[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:16:15 dist-test-slave-39l8 krb5kdc[20200](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:16:15 dist-test-slave-39l8 krb5kdc[20200](info): set up 2 sockets
Aug 09 12:16:15 dist-test-slave-39l8 krb5kdc[20200](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 12:16:16.406798 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.474s user 0.004s sys 0.024s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260809 12:16:16.533623 20191 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.126s user 0.003s sys 0.004s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260809 12:16:16.675561 20191 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.142s user 0.000s sys 0.006s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260809 12:16:16.795068 20191 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.119s user 0.000s sys 0.006s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/test-user.keytab.
W20260809 12:16:17.007725 20191 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.212s user 0.000s sys 0.007s
Aug 09 12:16:17 dist-test-slave-39l8 krb5kdc[20200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:16:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:16:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:16:17.104712 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:40323
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:40323
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:17.669245 20214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:17.669694 20214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:17.669926 20214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:17.679765 20214 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:16:17.680217 20214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:17.680357 20214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:17.680485 20214 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:16:17.680591 20214 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:16:17.692658 20214 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:40323
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:40323
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:17.694485 20214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:17.696367 20214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:17.716476 20224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:17.716753 20226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:17.716784 20228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:17.718152 20214 server_base.cc:1034] running on GCE node
I20260809 12:16:17.719280 20214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:17.721357 20214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:17.722611 20214 hybrid_clock.cc:648] HybridClock initialized: now 1786277777722521 us; error 83 us; skew 500 ppm
I20260809 12:16:17.723289 20214 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:17 dist-test-slave-39l8 krb5kdc[20200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277777, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:17.727545 20214 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:16:17.729991 20214 webserver.cc:460] Webserver started at http://127.19.183.254:36101/ using document root <none> and password file <none>
I20260809 12:16:17.731248 20214 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:17.731439 20214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:17.731904 20214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:17.736292 20214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "507dd4c6a45c4a52b0e64fb745252556"
format_stamp: "Formatted at 2026-08-09 12:16:17 on dist-test-slave-39l8"
server_key: "de2bb3c8913d04fb9f10bfb5d0a2b275"
server_key_iv: "e5e7b5feb5a583630f0fd0526fa58687"
server_key_version: "encryptionkey@0"
I20260809 12:16:17.737149 20214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "507dd4c6a45c4a52b0e64fb745252556"
format_stamp: "Formatted at 2026-08-09 12:16:17 on dist-test-slave-39l8"
server_key: "de2bb3c8913d04fb9f10bfb5d0a2b275"
server_key_iv: "e5e7b5feb5a583630f0fd0526fa58687"
server_key_version: "encryptionkey@0"
I20260809 12:16:17.745863 20214 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.009s sys 0.000s
I20260809 12:16:17.752492 20236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:17.754463 20214 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:16:17.754769 20214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "507dd4c6a45c4a52b0e64fb745252556"
format_stamp: "Formatted at 2026-08-09 12:16:17 on dist-test-slave-39l8"
server_key: "de2bb3c8913d04fb9f10bfb5d0a2b275"
server_key_iv: "e5e7b5feb5a583630f0fd0526fa58687"
server_key_version: "encryptionkey@0"
I20260809 12:16:17.755055 20214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:17.777647 20214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:17.783643 20214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:17.784119 20214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:17.826886 20214 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:40323
I20260809 12:16:17.827016 20308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:40323 every 8 connection(s)
I20260809 12:16:17.829409 20214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:16:17.831176 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 20214
I20260809 12:16:17.831775 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:16:17.834183 20191 external_mini_cluster.cc:1428] Setting key f40199e2bb172ed1b53a959ffa88985f
I20260809 12:16:17.837543 20309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:17.857066 20309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 507dd4c6a45c4a52b0e64fb745252556: Bootstrap starting.
I20260809 12:16:17.863727 20309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 507dd4c6a45c4a52b0e64fb745252556: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:17.865988 20309 log.cc:824] T 00000000000000000000000000000000 P 507dd4c6a45c4a52b0e64fb745252556: Log is configured to *not* fsync() on all Append() calls
Aug 09 12:16:17 dist-test-slave-39l8 krb5kdc[20200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:17.870625 20309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 507dd4c6a45c4a52b0e64fb745252556: No bootstrap required, opened a new log
I20260809 12:16:17.882301 20309 raft_consensus.cc:348] T 00000000000000000000000000000000 P 507dd4c6a45c4a52b0e64fb745252556 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507dd4c6a45c4a52b0e64fb745252556" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40323 } }
I20260809 12:16:17.882805 20312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:17.840179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43120 (local address 127.19.183.254:40323)
0809 12:16:17.841938 (+ 1759us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:17.841959 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:17.843578 (+ 1619us) server_negotiation.cc:407] Connection header received
0809 12:16:17.855490 (+ 11912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:17.855514 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:17.856049 (+ 535us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:17.856483 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:17.860663 (+ 4180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:17.862352 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:17.864332 (+ 1980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:17.865222 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:17.865486 (+ 264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:17.872022 (+ 6536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:17.872111 (+ 89us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:17.872144 (+ 33us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:17.872184 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:17.876305 (+ 4121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:17.877264 (+ 959us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:17.877270 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:17.877276 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:17.877500 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:17.878059 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:17.878063 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:17.878083 (+ 20us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:17.878928 (+ 845us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:17.879537 (+ 609us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:17.880473 (+ 936us) server_negotiation.cc:299] Negotiation successful
0809 12:16:17.880915 (+ 442us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1245,"thread_start_us":769,"threads_started":1}
I20260809 12:16:17.883013 20309 raft_consensus.cc:374] T 00000000000000000000000000000000 P 507dd4c6a45c4a52b0e64fb745252556 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:17.883450 20309 raft_consensus.cc:729] T 00000000000000000000000000000000 P 507dd4c6a45c4a52b0e64fb745252556 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 507dd4c6a45c4a52b0e64fb745252556, State: Initialized, Role: FOLLOWER
I20260809 12:16:17.885015 20309 consensus_queue.cc:260] T 00000000000000000000000000000000 P 507dd4c6a45c4a52b0e64fb745252556 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507dd4c6a45c4a52b0e64fb745252556" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40323 } }
I20260809 12:16:17.885578 20309 raft_consensus.cc:388] T 00000000000000000000000000000000 P 507dd4c6a45c4a52b0e64fb745252556 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:16:17.885964 20309 raft_consensus.cc:482] T 00000000000000000000000000000000 P 507dd4c6a45c4a52b0e64fb745252556 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:16:17.886226 20309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 507dd4c6a45c4a52b0e64fb745252556 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:17.889617 20309 raft_consensus.cc:504] T 00000000000000000000000000000000 P 507dd4c6a45c4a52b0e64fb745252556 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507dd4c6a45c4a52b0e64fb745252556" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40323 } }
I20260809 12:16:17.890666 20309 leader_election.cc:302] T 00000000000000000000000000000000 P 507dd4c6a45c4a52b0e64fb745252556 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 507dd4c6a45c4a52b0e64fb745252556; no voters:
I20260809 12:16:17.891681 20309 leader_election.cc:288] T 00000000000000000000000000000000 P 507dd4c6a45c4a52b0e64fb745252556 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:16:17.892010 20316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 507dd4c6a45c4a52b0e64fb745252556 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:16:17.893446 20316 raft_consensus.cc:686] T 00000000000000000000000000000000 P 507dd4c6a45c4a52b0e64fb745252556 [term 1 LEADER]: Becoming Leader. State: Replica: 507dd4c6a45c4a52b0e64fb745252556, State: Running, Role: LEADER
I20260809 12:16:17.894495 20316 consensus_queue.cc:237] T 00000000000000000000000000000000 P 507dd4c6a45c4a52b0e64fb745252556 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507dd4c6a45c4a52b0e64fb745252556" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40323 } }
I20260809 12:16:17.895602 20309 sys_catalog.cc:564] T 00000000000000000000000000000000 P 507dd4c6a45c4a52b0e64fb745252556 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:16:17.902772 20319 sys_catalog.cc:455] T 00000000000000000000000000000000 P 507dd4c6a45c4a52b0e64fb745252556 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "507dd4c6a45c4a52b0e64fb745252556" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507dd4c6a45c4a52b0e64fb745252556" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40323 } } }
I20260809 12:16:17.903759 20319 sys_catalog.cc:458] T 00000000000000000000000000000000 P 507dd4c6a45c4a52b0e64fb745252556 [sys.catalog]: This master's current role is: LEADER
I20260809 12:16:17.904407 20322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 507dd4c6a45c4a52b0e64fb745252556 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 507dd4c6a45c4a52b0e64fb745252556. Latest consensus state: current_term: 1 leader_uuid: "507dd4c6a45c4a52b0e64fb745252556" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "507dd4c6a45c4a52b0e64fb745252556" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40323 } } }
I20260809 12:16:17.905428 20322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 507dd4c6a45c4a52b0e64fb745252556 [sys.catalog]: This master's current role is: LEADER
I20260809 12:16:17.909873 20331 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:16:17.920418 20331 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:16:17.936637 20331 catalog_manager.cc:1347] Generated new cluster ID: f8599f0c9da4487aad6585fe478bc5a3
I20260809 12:16:17.936873 20331 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:16:17.952907 20331 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:16:17.955268 20331 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:16:17.969587 20331 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 507dd4c6a45c4a52b0e64fb745252556: Generated new TSK 0
I20260809 12:16:17.970973 20331 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
I20260809 12:16:18.030570 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40323
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:18.632905 20351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:18.633563 20351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:18.633849 20351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:18.644184 20351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:18.644413 20351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:18.644676 20351 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:16:18.659354 20351 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40323
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:18.661279 20351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:18.663066 20351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:18.684417 20358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:18.686061 20359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:18.686125 20363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:18.686532 20351 server_base.cc:1034] running on GCE node
I20260809 12:16:18.687367 20351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:18.688625 20351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:18.689908 20351 hybrid_clock.cc:648] HybridClock initialized: now 1786277778689824 us; error 52 us; skew 500 ppm
I20260809 12:16:18.690410 20351 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:18 dist-test-slave-39l8 krb5kdc[20200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277778, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:18.697916 20351 init.cc:377] Logged in from keytab as kudu/127.19.183.193@KRBTEST.COM (short username kudu)
I20260809 12:16:18.700454 20351 webserver.cc:460] Webserver started at http://127.19.183.193:46019/ using document root <none> and password file <none>
I20260809 12:16:18.701325 20351 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:18.701534 20351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:18.701958 20351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:18.705981 20351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "cf6117419da1456babddbef4d95e02b0"
format_stamp: "Formatted at 2026-08-09 12:16:18 on dist-test-slave-39l8"
server_key: "a1c4d7c654c8e78494b4a62def03725f"
server_key_iv: "9b111bb420cef04f09cc70aecbd0bc93"
server_key_version: "encryptionkey@0"
I20260809 12:16:18.707012 20351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "cf6117419da1456babddbef4d95e02b0"
format_stamp: "Formatted at 2026-08-09 12:16:18 on dist-test-slave-39l8"
server_key: "a1c4d7c654c8e78494b4a62def03725f"
server_key_iv: "9b111bb420cef04f09cc70aecbd0bc93"
server_key_version: "encryptionkey@0"
I20260809 12:16:18.714331 20351 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.001s
I20260809 12:16:18.719872 20373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:18.721526 20351 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.002s
I20260809 12:16:18.721735 20351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "cf6117419da1456babddbef4d95e02b0"
format_stamp: "Formatted at 2026-08-09 12:16:18 on dist-test-slave-39l8"
server_key: "a1c4d7c654c8e78494b4a62def03725f"
server_key_iv: "9b111bb420cef04f09cc70aecbd0bc93"
server_key_version: "encryptionkey@0"
I20260809 12:16:18.721997 20351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:18.749264 20351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:18.755359 20351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:18.755820 20351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:18.757486 20351 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:16:18.760129 20351 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:18.760255 20351 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:18.760485 20351 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:18.760620 20351 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:18.851295 20351 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:34767
I20260809 12:16:18.851409 20534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:34767 every 8 connection(s)
I20260809 12:16:18.853495 20351 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:16:18.863265 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 20351
I20260809 12:16:18.863612 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:16:18.864753 20191 external_mini_cluster.cc:1428] Setting key 8beefdec7ee2cdaebe9e8c07c5295875
Aug 09 12:16:18 dist-test-slave-39l8 krb5kdc[20200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277778, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:18.887079 20538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:18.858328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:54819 (local address 127.19.183.254:40323)
0809 12:16:18.859201 (+ 873us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:18.859206 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:18.860892 (+ 1686us) server_negotiation.cc:407] Connection header received
0809 12:16:18.864236 (+ 3344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:18.864242 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:18.864368 (+ 126us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:18.864599 (+ 231us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:18.866884 (+ 2285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:18.867970 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:18.870101 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:18.870694 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:18.870888 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:18.876938 (+ 6050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:18.876947 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:18.876952 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:18.877009 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:18.882562 (+ 5553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:18.883587 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:18.883595 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:18.883602 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:18.883911 (+ 309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:18.884509 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:18.884518 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:18.884524 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:18.885061 (+ 537us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:18.885303 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:18.885932 (+ 629us) server_negotiation.cc:299] Negotiation successful
0809 12:16:18.886258 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":717,"thread_start_us":580,"threads_started":1}
I20260809 12:16:18.888034 20539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:18.859248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:40323 (local address 127.19.183.193:54819)
0809 12:16:18.860684 (+ 1436us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:18.860731 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:18.863887 (+ 3156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:18.864862 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:18.864876 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:18.865817 (+ 941us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:18.866638 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:18.866659 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:18.868190 (+ 1531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:18.868195 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:18.869937 (+ 1742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:18.869950 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:18.871135 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:18.871146 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:18.871495 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:18.872399 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:18.872422 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:18.876643 (+ 4221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:18.882795 (+ 6152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:18.882809 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:18.882827 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:18.883423 (+ 596us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:18.884028 (+ 605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:18.884031 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:18.884033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:18.884366 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:18.885196 (+ 830us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:18.885202 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:18.885513 (+ 311us) client_negotiation.cc:769] Sending connection context
0809 12:16:18.885830 (+ 317us) client_negotiation.cc:240] Negotiation successful
0809 12:16:18.886432 (+ 602us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1095,"thread_start_us":845,"threads_started":1}
I20260809 12:16:18.890262 20536 heartbeater.cc:347] Connected to a master server at 127.19.183.254:40323
I20260809 12:16:18.890869 20536 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.19.183.194@KRBTEST.COM; defaulting to no policy
I20260809 12:16:18.892014 20536 heartbeater.cc:511] Master 127.19.183.254:40323 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.194@KRBTEST.COM" created.
I20260809 12:16:18.896025 20261 ts_manager.cc:194] Registered new tserver with Master: cf6117419da1456babddbef4d95e02b0 (127.19.183.193:34767)
I20260809 12:16:18.898463 20261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:54819
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
I20260809 12:16:18.920400 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40323
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:19.565970 20545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:19.566313 20545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:19.566452 20545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:19.576185 20545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:19.576334 20545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:19.576479 20545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:16:19.588815 20545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40323
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:19.590400 20545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:19.592145 20545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:19.610329 20552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:19.610670 20555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:19.611033 20551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:19.612082 20545 server_base.cc:1034] running on GCE node
I20260809 12:16:19.615944 20545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:19.617048 20545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:19.618330 20545 hybrid_clock.cc:648] HybridClock initialized: now 1786277779618244 us; error 41 us; skew 500 ppm
I20260809 12:16:19.618824 20545 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:19 dist-test-slave-39l8 krb5kdc[20200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277779, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:19.623293 20545 init.cc:377] Logged in from keytab as kudu/127.19.183.194@KRBTEST.COM (short username kudu)
I20260809 12:16:19.625370 20545 webserver.cc:460] Webserver started at http://127.19.183.194:40399/ using document root <none> and password file <none>
I20260809 12:16:19.626216 20545 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:19.626412 20545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:19.626947 20545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:19.631085 20545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "66927c8e45b34fd1aaa523fc0d23c160"
format_stamp: "Formatted at 2026-08-09 12:16:19 on dist-test-slave-39l8"
server_key: "732c80b0b09453c16d4e96e1803cac42"
server_key_iv: "80513909efaac89375f852f1c72427ff"
server_key_version: "encryptionkey@0"
I20260809 12:16:19.632246 20545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "66927c8e45b34fd1aaa523fc0d23c160"
format_stamp: "Formatted at 2026-08-09 12:16:19 on dist-test-slave-39l8"
server_key: "732c80b0b09453c16d4e96e1803cac42"
server_key_iv: "80513909efaac89375f852f1c72427ff"
server_key_version: "encryptionkey@0"
I20260809 12:16:19.639020 20545 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260809 12:16:19.643697 20568 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:19.645159 20545 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:16:19.645359 20545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "66927c8e45b34fd1aaa523fc0d23c160"
format_stamp: "Formatted at 2026-08-09 12:16:19 on dist-test-slave-39l8"
server_key: "732c80b0b09453c16d4e96e1803cac42"
server_key_iv: "80513909efaac89375f852f1c72427ff"
server_key_version: "encryptionkey@0"
I20260809 12:16:19.645625 20545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:19.682629 20545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:19.687779 20545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:19.688076 20545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:19.689476 20545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:16:19.691401 20545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:19.691547 20545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:19.691748 20545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:19.691833 20545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:19.756290 20545 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:45975
I20260809 12:16:19.756402 20740 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:45975 every 8 connection(s)
I20260809 12:16:19.757826 20545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:16:19.767936 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 20545
I20260809 12:16:19.768162 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:16:19.768641 20191 external_mini_cluster.cc:1428] Setting key 5906aa9a9abe79eb4764bccbaa168668
Aug 09 12:16:19 dist-test-slave-39l8 krb5kdc[20200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277779, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:19.781137 20743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:19.760973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:55497 (local address 127.19.183.254:40323)
0809 12:16:19.761553 (+ 580us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:19.761560 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:19.762795 (+ 1235us) server_negotiation.cc:407] Connection header received
0809 12:16:19.764913 (+ 2118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:19.764918 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:19.765003 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:19.765145 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:19.766820 (+ 1675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:19.767743 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:19.769792 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:19.770346 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:19.770501 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:19.774392 (+ 3891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:19.774398 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:19.774402 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:19.774434 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:19.777802 (+ 3368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:19.778563 (+ 761us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:19.778572 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:19.778576 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:19.778785 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:19.779261 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:19.779266 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:19.779269 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:19.779584 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:19.779753 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:19.780458 (+ 705us) server_negotiation.cc:299] Negotiation successful
0809 12:16:19.780714 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":453,"thread_start_us":333,"threads_started":1}
I20260809 12:16:19.782661 20745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:19.761720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:40323 (local address 127.19.183.194:55497)
0809 12:16:19.762574 (+ 854us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:19.762615 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:19.764610 (+ 1995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:19.765397 (+ 787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:19.765404 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:19.765994 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:19.766625 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:19.766642 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:19.767929 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:19.767934 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:19.769646 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:19.769660 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:19.770628 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:19.770637 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:19.770933 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:19.771692 (+ 759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:19.771726 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:19.774163 (+ 2437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:19.777959 (+ 3796us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:19.777968 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:19.777984 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:19.778412 (+ 428us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:19.778903 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:19.778908 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:19.778912 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:19.779134 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:19.779729 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:19.779736 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:19.780078 (+ 342us) client_negotiation.cc:769] Sending connection context
0809 12:16:19.780361 (+ 283us) client_negotiation.cc:240] Negotiation successful
0809 12:16:19.780901 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":599,"thread_start_us":399,"threads_started":1}
I20260809 12:16:19.784437 20741 heartbeater.cc:347] Connected to a master server at 127.19.183.254:40323
I20260809 12:16:19.784997 20741 heartbeater.cc:464] Registering TS with master...
I20260809 12:16:19.786500 20741 heartbeater.cc:511] Master 127.19.183.254:40323 requested a full tablet report, sending...
I20260809 12:16:19.789072 20261 ts_manager.cc:194] Registered new tserver with Master: 66927c8e45b34fd1aaa523fc0d23c160 (127.19.183.194:45975)
I20260809 12:16:19.790434 20261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:55497
WARNING: no policy specified for kudu/127.19.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
I20260809 12:16:19.820340 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40323
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40209
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:16:19.902280 20536 heartbeater.cc:503] Master 127.19.183.254:40323 was elected leader, sending a full tablet report...
W20260809 12:16:20.432142 20749 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:20.432476 20749 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:20.432705 20749 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:20.447009 20749 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:20.447247 20749 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:20.447526 20749 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:16:20.460820 20749 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40209
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40323
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:20.462296 20749 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:20.463886 20749 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:20.483052 20756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:20.483307 20758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:20.483453 20763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:20.485020 20749 server_base.cc:1034] running on GCE node
I20260809 12:16:20.485853 20749 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:20.487128 20749 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:20.488443 20749 hybrid_clock.cc:648] HybridClock initialized: now 1786277780488318 us; error 50 us; skew 500 ppm
I20260809 12:16:20.488873 20749 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:20 dist-test-slave-39l8 krb5kdc[20200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277780, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:20.493144 20749 init.cc:377] Logged in from keytab as kudu/127.19.183.195@KRBTEST.COM (short username kudu)
I20260809 12:16:20.494879 20749 webserver.cc:460] Webserver started at http://127.19.183.195:43123/ using document root <none> and password file <none>
I20260809 12:16:20.495733 20749 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:20.495935 20749 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:20.496372 20749 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:20.499820 20749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "c9b2034c237a4203be0717b5bb0e9983"
format_stamp: "Formatted at 2026-08-09 12:16:20 on dist-test-slave-39l8"
server_key: "9793e0da69e098f3cd8a3b14da1b144c"
server_key_iv: "68bcedec950d8548454df3ad77930457"
server_key_version: "encryptionkey@0"
I20260809 12:16:20.500695 20749 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "c9b2034c237a4203be0717b5bb0e9983"
format_stamp: "Formatted at 2026-08-09 12:16:20 on dist-test-slave-39l8"
server_key: "9793e0da69e098f3cd8a3b14da1b144c"
server_key_iv: "68bcedec950d8548454df3ad77930457"
server_key_version: "encryptionkey@0"
I20260809 12:16:20.506604 20749 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:16:20.511444 20774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:20.512892 20749 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 12:16:20.513118 20749 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "c9b2034c237a4203be0717b5bb0e9983"
format_stamp: "Formatted at 2026-08-09 12:16:20 on dist-test-slave-39l8"
server_key: "9793e0da69e098f3cd8a3b14da1b144c"
server_key_iv: "68bcedec950d8548454df3ad77930457"
server_key_version: "encryptionkey@0"
I20260809 12:16:20.513367 20749 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:20.532459 20749 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:20.537458 20749 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:20.537961 20749 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:20.539552 20749 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:16:20.541425 20749 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:20.541610 20749 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:20.541803 20749 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:20.541904 20749 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:20.613760 20749 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:39205
I20260809 12:16:20.613948 20928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:39205 every 8 connection(s)
I20260809 12:16:20.615533 20749 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:16:20.619036 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 20749
I20260809 12:16:20.619302 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:16:20.620052 20191 external_mini_cluster.cc:1428] Setting key bdb9caf043cab2d9e7a0113ef0313e66
Aug 09 12:16:20 dist-test-slave-39l8 krb5kdc[20200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277780, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:20.643196 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:20.618768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:41851 (local address 127.19.183.254:40323)
0809 12:16:20.619369 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:20.619375 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:20.621205 (+ 1830us) server_negotiation.cc:407] Connection header received
0809 12:16:20.624037 (+ 2832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:20.624042 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:20.624218 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:20.624443 (+ 225us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:20.626772 (+ 2329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:20.628370 (+ 1598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:20.630498 (+ 2128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:20.631066 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:20.631170 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:20.635463 (+ 4293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:20.635469 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:20.635473 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:20.635505 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:20.639822 (+ 4317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:20.640608 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:20.640612 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:20.640614 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:20.640804 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:20.641380 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:20.641384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:20.641386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:20.641769 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:20.641943 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:20.642579 (+ 636us) server_negotiation.cc:299] Negotiation successful
0809 12:16:20.642831 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465,"thread_start_us":300,"threads_started":1}
I20260809 12:16:20.644364 20933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:20.619530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:40323 (local address 127.19.183.195:41851)
0809 12:16:20.620890 (+ 1360us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:20.620957 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:20.623691 (+ 2734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:20.624787 (+ 1096us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:20.624799 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:20.625778 (+ 979us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:20.626564 (+ 786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:20.626583 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:20.628533 (+ 1950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:20.628537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:20.630347 (+ 1810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:20.630360 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:20.631349 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:20.631356 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:20.631628 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:20.632437 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:20.632459 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:20.635202 (+ 2743us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:20.639967 (+ 4765us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:20.639973 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:20.639985 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:20.640503 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:20.640911 (+ 408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:20.640914 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:20.640916 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:20.641164 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:20.641911 (+ 747us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:20.641917 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:20.642234 (+ 317us) client_negotiation.cc:769] Sending connection context
0809 12:16:20.642497 (+ 263us) client_negotiation.cc:240] Negotiation successful
0809 12:16:20.642997 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1010,"thread_start_us":661,"threads_started":1}
I20260809 12:16:20.645787 20930 heartbeater.cc:347] Connected to a master server at 127.19.183.254:40323
I20260809 12:16:20.646075 20930 heartbeater.cc:464] Registering TS with master...
I20260809 12:16:20.646971 20930 heartbeater.cc:511] Master 127.19.183.254:40323 requested a full tablet report, sending...
I20260809 12:16:20.648693 20261 ts_manager.cc:194] Registered new tserver with Master: c9b2034c237a4203be0717b5bb0e9983 (127.19.183.195:39205)
I20260809 12:16:20.649803 20261 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:41851
I20260809 12:16:20.651382 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 12:16:20 dist-test-slave-39l8 krb5kdc[20200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277780, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 12:16:20 dist-test-slave-39l8 krb5kdc[20200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277780, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.183.193@KRBTEST.COM
I20260809 12:16:20.711887 20941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:20.690803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56258 (local address 127.19.183.193:34767)
0809 12:16:20.691939 (+ 1136us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:20.691947 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:20.691979 (+ 32us) server_negotiation.cc:407] Connection header received
0809 12:16:20.692233 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:20.692240 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:20.692475 (+ 235us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:20.692610 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:20.694813 (+ 2203us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:20.696273 (+ 1460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:20.698109 (+ 1836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:20.698867 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:20.698968 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:20.704337 (+ 5369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:20.704349 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:20.704363 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:20.704423 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:20.708710 (+ 4287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:20.709442 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:20.709450 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:20.709452 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:20.709613 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:20.709990 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:20.709993 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:20.709995 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:20.710397 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:20.710665 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:20.710980 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 12:16:20.711245 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":974,"thread_start_us":782,"threads_started":1}
W20260809 12:16:20.713428 20415 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56258
Aug 09 12:16:20 dist-test-slave-39l8 krb5kdc[20200](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 12:16:20 dist-test-slave-39l8 krb5kdc[20200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.193@KRBTEST.COM
I20260809 12:16:20.769929 20941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:20.752827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56274 (local address 127.19.183.193:34767)
0809 12:16:20.753117 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:20.753124 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:20.753300 (+ 176us) server_negotiation.cc:407] Connection header received
0809 12:16:20.753463 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:20.753467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:20.753594 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:20.753685 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:20.755195 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:20.756092 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:20.757482 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:20.758046 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:20.758233 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:20.763346 (+ 5113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:20.763352 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:20.763355 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:20.763392 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:20.766943 (+ 3551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:20.767756 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:20.767762 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:20.767765 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:20.767953 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:20.768404 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:20.768409 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:20.768414 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:20.768714 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:20.768991 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:20.769299 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 12:16:20.769550 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103}
I20260809 12:16:20.771914 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 20351
I20260809 12:16:20.787824 20941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:20.773059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56288 (local address 127.19.183.193:34767)
0809 12:16:20.773316 (+ 257us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:20.773323 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:20.773661 (+ 338us) server_negotiation.cc:407] Connection header received
0809 12:16:20.773840 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:20.773851 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:20.773907 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:20.774052 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:20.775472 (+ 1420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:20.776177 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:20.777908 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:20.778531 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:20.778676 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:20.781654 (+ 2978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:20.781659 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:20.781663 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:20.781692 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:20.784773 (+ 3081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:20.785503 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:20.785516 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:20.785518 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:20.785680 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:20.786103 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:20.786110 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:20.786116 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:20.786533 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:20.786904 (+ 371us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:20.787196 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 12:16:20.787408 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 12:16:20.788662 20527 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:16:20.793915 20741 heartbeater.cc:503] Master 127.19.183.254:40323 was elected leader, sending a full tablet report...
I20260809 12:16:20.878544 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 20351
I20260809 12:16:20.903723 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 20545
Aug 09 12:16:20 dist-test-slave-39l8 krb5kdc[20200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.194@KRBTEST.COM
I20260809 12:16:20.920186 20954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:20.904705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54712 (local address 127.19.183.194:45975)
0809 12:16:20.905413 (+ 708us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:20.905420 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:20.905447 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:16:20.905512 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:20.905518 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:20.905775 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:20.905919 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:20.907249 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:20.908211 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:20.909418 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:20.909955 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:20.910048 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:20.914196 (+ 4148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:20.914210 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:20.914225 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:20.914264 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:20.917343 (+ 3079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:20.918010 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:20.918018 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:20.918020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:20.918148 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:20.918615 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:20.918624 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:20.918626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:20.919211 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:20.919456 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:20.919705 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 12:16:20.919901 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":401,"threads_started":1}
I20260809 12:16:20.921470 20736 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:16:21.004637 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 20545
I20260809 12:16:21.030896 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 20749
Aug 09 12:16:21 dist-test-slave-39l8 krb5kdc[20200](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277780, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.195@KRBTEST.COM
I20260809 12:16:21.047011 20956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:21.032072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59792 (local address 127.19.183.195:39205)
0809 12:16:21.032611 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:21.032619 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:21.032649 (+ 30us) server_negotiation.cc:407] Connection header received
0809 12:16:21.032712 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:21.032718 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:21.032927 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:21.033053 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:21.034310 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:21.035458 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:21.036673 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:21.037212 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:21.037307 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:21.041329 (+ 4022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:21.041341 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:21.041355 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:21.041393 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:21.044472 (+ 3079us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:21.045087 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:21.045109 (+ 22us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:21.045111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:21.045236 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:21.045548 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:21.045552 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:21.045553 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:21.045891 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:21.046148 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:21.046359 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 12:16:21.046556 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":229,"threads_started":1}
I20260809 12:16:21.048331 20924 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:16:21.131731 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 20749
I20260809 12:16:21.155380 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 20214
I20260809 12:16:21.156770 20304 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:16:21.261830 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 20214
2026-08-09T12:16:21Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (6401 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:16:21 dist-test-slave-39l8 krb5kdc[20962](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:16:21 dist-test-slave-39l8 krb5kdc[20962](info): set up 2 sockets
Aug 09 12:16:21 dist-test-slave-39l8 krb5kdc[20962](info): commencing operation
krb5kdc: starting...
W20260809 12:16:22.566844 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.253s user 0.000s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:16:22 dist-test-slave-39l8 krb5kdc[20962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:16:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:16:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:16:22.763504 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:39447
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:33297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:39447
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:23.090000 20980 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:23.090298 20980 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:23.090456 20980 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:23.097620 20980 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:16:23.097750 20980 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:23.097836 20980 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:23.097934 20980 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:16:23.097993 20980 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:16:23.107604 20980 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:33297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:39447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:39447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:23.108870 20980 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:23.110226 20980 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:23.122051 20989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:23.122082 20988 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:23.122149 20991 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:23.123008 20980 server_base.cc:1034] running on GCE node
I20260809 12:16:23.123688 20980 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:23.124977 20980 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:23.126186 20980 hybrid_clock.cc:648] HybridClock initialized: now 1786277783126145 us; error 34 us; skew 500 ppm
I20260809 12:16:23.126550 20980 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:23 dist-test-slave-39l8 krb5kdc[20962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277783, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:23.129684 20980 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:16:23.131163 20980 webserver.cc:460] Webserver started at http://127.19.183.254:43987/ using document root <none> and password file <none>
I20260809 12:16:23.131824 20980 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:23.131932 20980 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:23.132252 20980 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:23.135507 20980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "4b673022846a43048e307d47f9a56ef7"
format_stamp: "Formatted at 2026-08-09 12:16:23 on dist-test-slave-39l8"
server_key: "19f8ceecf952631da53ab55e42ba80d7"
server_key_iv: "9d7fb7d1b8375b65d13fd9d9f467d076"
server_key_version: "encryptionkey@0"
I20260809 12:16:23.136256 20980 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "4b673022846a43048e307d47f9a56ef7"
format_stamp: "Formatted at 2026-08-09 12:16:23 on dist-test-slave-39l8"
server_key: "19f8ceecf952631da53ab55e42ba80d7"
server_key_iv: "9d7fb7d1b8375b65d13fd9d9f467d076"
server_key_version: "encryptionkey@0"
I20260809 12:16:23.141587 20980 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 12:16:23.145025 21000 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:23.146373 20980 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 12:16:23.146597 20980 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "4b673022846a43048e307d47f9a56ef7"
format_stamp: "Formatted at 2026-08-09 12:16:23 on dist-test-slave-39l8"
server_key: "19f8ceecf952631da53ab55e42ba80d7"
server_key_iv: "9d7fb7d1b8375b65d13fd9d9f467d076"
server_key_version: "encryptionkey@0"
I20260809 12:16:23.146876 20980 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:23.170674 20980 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:23.175148 20980 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:23.175480 20980 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:23.196808 20980 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:39447
I20260809 12:16:23.196812 21071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:39447 every 8 connection(s)
I20260809 12:16:23.198302 20980 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:16:23.200644 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 20980
I20260809 12:16:23.200903 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:16:23.201660 20191 external_mini_cluster.cc:1428] Setting key 33d2e4c6d37849378f109f746890aafd
I20260809 12:16:23.202292 21073 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:23.217381 21073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b673022846a43048e307d47f9a56ef7: Bootstrap starting.
Aug 09 12:16:23 dist-test-slave-39l8 krb5kdc[20962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:23.221279 21073 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4b673022846a43048e307d47f9a56ef7: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:23.222401 21073 log.cc:824] T 00000000000000000000000000000000 P 4b673022846a43048e307d47f9a56ef7: Log is configured to *not* fsync() on all Append() calls
I20260809 12:16:23.225718 21073 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4b673022846a43048e307d47f9a56ef7: No bootstrap required, opened a new log
I20260809 12:16:23.226912 21079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:23.203818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56532 (local address 127.19.183.254:39447)
0809 12:16:23.204808 (+ 990us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:23.204820 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:23.204863 (+ 43us) server_negotiation.cc:407] Connection header received
0809 12:16:23.207437 (+ 2574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:23.207460 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:23.207894 (+ 434us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:23.208256 (+ 362us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:23.210119 (+ 1863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:23.211648 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:23.213543 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:23.214246 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:23.214401 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:23.219537 (+ 5136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:23.219545 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:23.219558 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:23.219593 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:23.222787 (+ 3194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:23.223528 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:23.223539 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:23.223552 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:23.223778 (+ 226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:23.224200 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:23.224203 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:23.224205 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:23.224720 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:23.224935 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:23.225509 (+ 574us) server_negotiation.cc:299] Negotiation successful
0809 12:16:23.225876 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":527,"threads_started":1}
I20260809 12:16:23.234623 21073 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4b673022846a43048e307d47f9a56ef7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b673022846a43048e307d47f9a56ef7" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39447 } }
I20260809 12:16:23.235067 21073 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4b673022846a43048e307d47f9a56ef7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:23.235193 21073 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4b673022846a43048e307d47f9a56ef7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b673022846a43048e307d47f9a56ef7, State: Initialized, Role: FOLLOWER
I20260809 12:16:23.235816 21073 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4b673022846a43048e307d47f9a56ef7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b673022846a43048e307d47f9a56ef7" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39447 } }
I20260809 12:16:23.236059 21073 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4b673022846a43048e307d47f9a56ef7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:16:23.236197 21073 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4b673022846a43048e307d47f9a56ef7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:16:23.236368 21073 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4b673022846a43048e307d47f9a56ef7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:23.238621 21073 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4b673022846a43048e307d47f9a56ef7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b673022846a43048e307d47f9a56ef7" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39447 } }
I20260809 12:16:23.239260 21073 leader_election.cc:302] T 00000000000000000000000000000000 P 4b673022846a43048e307d47f9a56ef7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4b673022846a43048e307d47f9a56ef7; no voters:
I20260809 12:16:23.239929 21073 leader_election.cc:288] T 00000000000000000000000000000000 P 4b673022846a43048e307d47f9a56ef7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:16:23.240020 21082 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4b673022846a43048e307d47f9a56ef7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:16:23.240914 21082 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4b673022846a43048e307d47f9a56ef7 [term 1 LEADER]: Becoming Leader. State: Replica: 4b673022846a43048e307d47f9a56ef7, State: Running, Role: LEADER
I20260809 12:16:23.241547 21082 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4b673022846a43048e307d47f9a56ef7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b673022846a43048e307d47f9a56ef7" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39447 } }
I20260809 12:16:23.242250 21073 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4b673022846a43048e307d47f9a56ef7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:16:23.244135 21084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b673022846a43048e307d47f9a56ef7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4b673022846a43048e307d47f9a56ef7. Latest consensus state: current_term: 1 leader_uuid: "4b673022846a43048e307d47f9a56ef7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b673022846a43048e307d47f9a56ef7" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39447 } } }
I20260809 12:16:23.244537 21084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b673022846a43048e307d47f9a56ef7 [sys.catalog]: This master's current role is: LEADER
I20260809 12:16:23.244369 21083 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4b673022846a43048e307d47f9a56ef7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4b673022846a43048e307d47f9a56ef7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4b673022846a43048e307d47f9a56ef7" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39447 } } }
I20260809 12:16:23.245002 21083 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4b673022846a43048e307d47f9a56ef7 [sys.catalog]: This master's current role is: LEADER
I20260809 12:16:23.252132 21089 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:16:23.257237 21089 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:16:23.266745 21089 catalog_manager.cc:1347] Generated new cluster ID: b47aee40ad4245d9932e6b117362b65e
I20260809 12:16:23.266866 21089 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:16:23.285904 21089 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:16:23.287212 21089 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:16:23.292457 21089 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4b673022846a43048e307d47f9a56ef7: Generated new TSK 0
I20260809 12:16:23.293197 21089 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
I20260809 12:16:23.354350 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:39447
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:33297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:23.731447 21115 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:23.731691 21115 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:23.731784 21115 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:23.738586 21115 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:23.738691 21115 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:23.738862 21115 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:16:23.748342 21115 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:33297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:39447
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:23.749596 21115 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:23.750887 21115 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:23.767136 21125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:23.767370 21126 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:23.767889 21128 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:23.768852 21115 server_base.cc:1034] running on GCE node
I20260809 12:16:23.769443 21115 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:23.770309 21115 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:23.771533 21115 hybrid_clock.cc:648] HybridClock initialized: now 1786277783771496 us; error 50 us; skew 500 ppm
I20260809 12:16:23.771864 21115 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:23 dist-test-slave-39l8 krb5kdc[20962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277783, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:23.780617 21115 init.cc:377] Logged in from keytab as kudu/127.19.183.193@KRBTEST.COM (short username kudu)
I20260809 12:16:23.782182 21115 webserver.cc:460] Webserver started at http://127.19.183.193:34261/ using document root <none> and password file <none>
I20260809 12:16:23.782955 21115 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:23.783102 21115 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:23.783414 21115 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:23.786293 21115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "a43e59570f27491db98013483342326f"
format_stamp: "Formatted at 2026-08-09 12:16:23 on dist-test-slave-39l8"
server_key: "dbe02d8d6fefd7607890308eaf804f4f"
server_key_iv: "e83f39dc3434cbb3836e22041197cf67"
server_key_version: "encryptionkey@0"
I20260809 12:16:23.787107 21115 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "a43e59570f27491db98013483342326f"
format_stamp: "Formatted at 2026-08-09 12:16:23 on dist-test-slave-39l8"
server_key: "dbe02d8d6fefd7607890308eaf804f4f"
server_key_iv: "e83f39dc3434cbb3836e22041197cf67"
server_key_version: "encryptionkey@0"
I20260809 12:16:23.792073 21115 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 12:16:23.795720 21143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:23.796973 21115 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:16:23.797168 21115 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "a43e59570f27491db98013483342326f"
format_stamp: "Formatted at 2026-08-09 12:16:23 on dist-test-slave-39l8"
server_key: "dbe02d8d6fefd7607890308eaf804f4f"
server_key_iv: "e83f39dc3434cbb3836e22041197cf67"
server_key_version: "encryptionkey@0"
I20260809 12:16:23.797361 21115 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:23.820061 21115 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:23.824038 21115 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:23.824365 21115 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:23.825342 21115 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:16:23.826665 21115 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:23.826797 21115 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 12:16:23.826932 21115 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:23.827003 21115 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:23.861228 21115 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:39681
I20260809 12:16:23.861287 21314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:39681 every 8 connection(s)
I20260809 12:16:23.862687 21115 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:16:23.868882 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 21115
I20260809 12:16:23.869117 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:16:23.869760 20191 external_mini_cluster.cc:1428] Setting key f1ca07a745c5fd4a52ba1aa485aa6565
Aug 09 12:16:23 dist-test-slave-39l8 krb5kdc[20962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277783, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:23.883405 21318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:23.865332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:39331 (local address 127.19.183.254:39447)
0809 12:16:23.865819 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:23.865823 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:23.866641 (+ 818us) server_negotiation.cc:407] Connection header received
0809 12:16:23.868553 (+ 1912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:23.868561 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:23.868655 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:23.868786 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:23.870437 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:23.871260 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:23.872969 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:23.873840 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:23.874009 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:23.877348 (+ 3339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:23.877357 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:23.877364 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:23.877406 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:23.880208 (+ 2802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:23.881158 (+ 950us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:23.881162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:23.881164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:23.881279 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:23.881757 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:23.881760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:23.881762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:23.881994 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:23.882111 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:23.882992 (+ 881us) server_negotiation.cc:299] Negotiation successful
0809 12:16:23.883165 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":268,"threads_started":1}
I20260809 12:16:23.884940 21319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:23.865728 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39447 (local address 127.19.183.193:39331)
0809 12:16:23.866458 (+ 730us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:23.866524 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:23.868283 (+ 1759us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:23.869040 (+ 757us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:23.869052 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:23.869632 (+ 580us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:23.870232 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:23.870254 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:23.871434 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:23.871458 (+ 24us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:23.872814 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:23.872827 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:23.874073 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:23.874078 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:23.874308 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:23.874990 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:23.875012 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:23.877124 (+ 2112us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:23.880373 (+ 3249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:23.880385 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:23.880401 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:23.881035 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:23.881384 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:23.881392 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:23.881398 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:23.881649 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:23.882130 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:23.882141 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:23.882562 (+ 421us) client_negotiation.cc:769] Sending connection context
0809 12:16:23.882907 (+ 345us) client_negotiation.cc:240] Negotiation successful
0809 12:16:23.883479 (+ 572us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":283,"threads_started":1}
I20260809 12:16:23.886514 21316 heartbeater.cc:347] Connected to a master server at 127.19.183.254:39447
I20260809 12:16:23.886878 21316 heartbeater.cc:464] Registering TS with master...
I20260809 12:16:23.887593 21316 heartbeater.cc:511] Master 127.19.183.254:39447 requested a full tablet report, sending...
I20260809 12:16:23.889823 21021 ts_manager.cc:194] Registered new tserver with Master: a43e59570f27491db98013483342326f (127.19.183.193:39681)
I20260809 12:16:23.891301 21021 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:39331
WARNING: no policy specified for kudu/127.19.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
I20260809 12:16:23.917729 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:39447
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:33297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:24.255111 21323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:24.255415 21323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:24.255564 21323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:24.262895 21323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:24.263028 21323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:24.263212 21323 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:16:24.272758 21323 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:33297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:39447
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:24.274103 21323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:24.275518 21323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:24.288902 21338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:24.288966 21335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:24.288990 21333 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:24.291999 21323 server_base.cc:1034] running on GCE node
I20260809 12:16:24.292642 21323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:24.293577 21323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:24.294857 21323 hybrid_clock.cc:648] HybridClock initialized: now 1786277784294803 us; error 41 us; skew 500 ppm
I20260809 12:16:24.295274 21323 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:24 dist-test-slave-39l8 krb5kdc[20962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277784, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:24.298442 21323 init.cc:377] Logged in from keytab as kudu/127.19.183.194@KRBTEST.COM (short username kudu)
I20260809 12:16:24.300197 21323 webserver.cc:460] Webserver started at http://127.19.183.194:34799/ using document root <none> and password file <none>
I20260809 12:16:24.301026 21323 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:24.301182 21323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:24.301507 21323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:24.304490 21323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "4b6b40d09acd46e99f02d9f16f0a1635"
format_stamp: "Formatted at 2026-08-09 12:16:24 on dist-test-slave-39l8"
server_key: "f17af749c52fd4dbc9a7dd5b64fa4209"
server_key_iv: "f45f20bd8522c6ba9985f05e4ff1c975"
server_key_version: "encryptionkey@0"
I20260809 12:16:24.305325 21323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "4b6b40d09acd46e99f02d9f16f0a1635"
format_stamp: "Formatted at 2026-08-09 12:16:24 on dist-test-slave-39l8"
server_key: "f17af749c52fd4dbc9a7dd5b64fa4209"
server_key_iv: "f45f20bd8522c6ba9985f05e4ff1c975"
server_key_version: "encryptionkey@0"
I20260809 12:16:24.310351 21323 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 12:16:24.314174 21348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:24.315541 21323 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 12:16:24.315755 21323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "4b6b40d09acd46e99f02d9f16f0a1635"
format_stamp: "Formatted at 2026-08-09 12:16:24 on dist-test-slave-39l8"
server_key: "f17af749c52fd4dbc9a7dd5b64fa4209"
server_key_iv: "f45f20bd8522c6ba9985f05e4ff1c975"
server_key_version: "encryptionkey@0"
I20260809 12:16:24.316007 21323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:24.372730 21323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:24.376690 21323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:24.377025 21323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:24.378209 21323 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:16:24.379772 21323 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:24.379882 21323 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:24.380038 21323 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:24.380117 21323 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:24.417063 21323 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:34439
I20260809 12:16:24.417152 21504 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:34439 every 8 connection(s)
I20260809 12:16:24.418392 21323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:16:24.428495 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 21323
I20260809 12:16:24.428714 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:16:24.429286 20191 external_mini_cluster.cc:1428] Setting key db50dd63ef05fef1e38df7714ed06823
Aug 09 12:16:24 dist-test-slave-39l8 krb5kdc[20962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277784, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:24.441354 21511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:24.421075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:42507 (local address 127.19.183.254:39447)
0809 12:16:24.421543 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:24.421549 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:24.422509 (+ 960us) server_negotiation.cc:407] Connection header received
0809 12:16:24.424383 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:24.424387 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:24.424457 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:24.424554 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:24.426000 (+ 1446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:24.426693 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:24.428242 (+ 1549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:24.428961 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:24.429086 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:24.433319 (+ 4233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:24.433326 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:24.433340 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:24.433407 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:24.437837 (+ 4430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:24.438678 (+ 841us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:24.438686 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:24.438692 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:24.438897 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:24.439419 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:24.439426 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:24.439431 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:24.439871 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:24.440029 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:24.440634 (+ 605us) server_negotiation.cc:299] Negotiation successful
0809 12:16:24.440878 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":344,"thread_start_us":222,"threads_started":1}
I20260809 12:16:24.442121 21513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:24.421666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39447 (local address 127.19.183.194:42507)
0809 12:16:24.422340 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:24.422380 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:24.424113 (+ 1733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:24.424783 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:24.424791 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:24.425314 (+ 523us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:24.425851 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:24.425867 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:24.426885 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:24.426889 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:24.428068 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:24.428080 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:24.429165 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:24.429170 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:24.429354 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:24.430201 (+ 847us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:24.430234 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:24.432919 (+ 2685us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:24.438055 (+ 5136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:24.438061 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:24.438088 (+ 27us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:24.438480 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:24.439125 (+ 645us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:24.439129 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:24.439131 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:24.439302 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:24.440004 (+ 702us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:24.440009 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:24.440283 (+ 274us) client_negotiation.cc:769] Sending connection context
0809 12:16:24.440495 (+ 212us) client_negotiation.cc:240] Negotiation successful
0809 12:16:24.440855 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":260,"threads_started":1}
I20260809 12:16:24.443742 21507 heartbeater.cc:347] Connected to a master server at 127.19.183.254:39447
I20260809 12:16:24.444088 21507 heartbeater.cc:464] Registering TS with master...
I20260809 12:16:24.444790 21507 heartbeater.cc:511] Master 127.19.183.254:39447 requested a full tablet report, sending...
I20260809 12:16:24.446416 21021 ts_manager.cc:194] Registered new tserver with Master: 4b6b40d09acd46e99f02d9f16f0a1635 (127.19.183.194:34439)
I20260809 12:16:24.447769 21021 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:42507
WARNING: no policy specified for kudu/127.19.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
I20260809 12:16:24.479141 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:39447
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:33297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:24.827318 21516 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:24.827580 21516 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:24.827730 21516 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:24.834657 21516 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:24.834767 21516 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:24.834933 21516 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:16:24.844172 21516 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:33297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:39447
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:24.845453 21516 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:24.846956 21516 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:24.860409 21523 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:24.860368 21526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:24.860317 21528 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:24.866257 21516 server_base.cc:1034] running on GCE node
I20260809 12:16:24.866914 21516 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:24.867805 21516 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:24.869036 21516 hybrid_clock.cc:648] HybridClock initialized: now 1786277784868994 us; error 63 us; skew 500 ppm
I20260809 12:16:24.869343 21516 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:24 dist-test-slave-39l8 krb5kdc[20962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277784, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:24.872503 21516 init.cc:377] Logged in from keytab as kudu/127.19.183.195@KRBTEST.COM (short username kudu)
I20260809 12:16:24.873872 21516 webserver.cc:460] Webserver started at http://127.19.183.195:35137/ using document root <none> and password file <none>
I20260809 12:16:24.874485 21516 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:24.874578 21516 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:24.874887 21516 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:24.877867 21516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "23786868badc4d9a886244bd2b962b76"
format_stamp: "Formatted at 2026-08-09 12:16:24 on dist-test-slave-39l8"
server_key: "ebacc529a53a6a7b6ea5fa11ce1cc8f3"
server_key_iv: "341945b6fa4214ab82a32a73758a34b0"
server_key_version: "encryptionkey@0"
I20260809 12:16:24.878665 21516 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "23786868badc4d9a886244bd2b962b76"
format_stamp: "Formatted at 2026-08-09 12:16:24 on dist-test-slave-39l8"
server_key: "ebacc529a53a6a7b6ea5fa11ce1cc8f3"
server_key_iv: "341945b6fa4214ab82a32a73758a34b0"
server_key_version: "encryptionkey@0"
I20260809 12:16:24.884159 21516 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 12:16:24.888113 21536 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:24.889384 21516 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:16:24.889518 21516 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "23786868badc4d9a886244bd2b962b76"
format_stamp: "Formatted at 2026-08-09 12:16:24 on dist-test-slave-39l8"
server_key: "ebacc529a53a6a7b6ea5fa11ce1cc8f3"
server_key_iv: "341945b6fa4214ab82a32a73758a34b0"
server_key_version: "encryptionkey@0"
I20260809 12:16:24.889694 21516 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:24.894446 21316 heartbeater.cc:503] Master 127.19.183.254:39447 was elected leader, sending a full tablet report...
I20260809 12:16:24.908277 21516 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:24.913420 21516 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:24.913771 21516 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:24.914855 21516 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:16:24.916672 21516 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:24.916855 21516 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:24.917088 21516 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:24.917203 21516 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:24.978030 21516 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:36771
I20260809 12:16:24.978209 21702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:36771 every 8 connection(s)
I20260809 12:16:24.979604 21516 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:16:24.988545 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 21516
I20260809 12:16:24.988797 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:16:24.989269 20191 external_mini_cluster.cc:1428] Setting key c186ef038f104051448fd03be436e2d9
Aug 09 12:16:24 dist-test-slave-39l8 krb5kdc[20962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277784, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:25.005959 21707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:24.982883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:57517 (local address 127.19.183.254:39447)
0809 12:16:24.983585 (+ 702us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:24.983592 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:24.984968 (+ 1376us) server_negotiation.cc:407] Connection header received
0809 12:16:24.987522 (+ 2554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:24.987527 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:24.987619 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:24.987716 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:24.989788 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:24.991156 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:24.993183 (+ 2027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:24.993666 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:24.993825 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:24.998417 (+ 4592us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:24.998422 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:24.998426 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:24.998478 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:25.001895 (+ 3417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:25.002958 (+ 1063us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:25.002964 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:25.002968 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:25.003159 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:25.003707 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:25.003716 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:25.003719 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:25.004053 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:25.004275 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:25.005076 (+ 801us) server_negotiation.cc:299] Negotiation successful
0809 12:16:25.005291 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":575,"thread_start_us":409,"threads_started":1}
I20260809 12:16:25.007083 21708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:24.983695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39447 (local address 127.19.183.195:57517)
0809 12:16:24.984819 (+ 1124us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:24.984857 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:24.987284 (+ 2427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:24.987917 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:24.987924 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:24.988704 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:24.989595 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:24.989614 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:24.991305 (+ 1691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:24.991310 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:24.993023 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:24.993036 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:24.993986 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:24.993995 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:24.994339 (+ 344us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:24.995055 (+ 716us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:24.995078 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:24.998168 (+ 3090us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:25.002163 (+ 3995us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:25.002172 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:25.002192 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:25.002821 (+ 629us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:25.003284 (+ 463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:25.003289 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:25.003293 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:25.003566 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:25.004192 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:25.004201 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:25.004618 (+ 417us) client_negotiation.cc:769] Sending connection context
0809 12:16:25.004893 (+ 275us) client_negotiation.cc:240] Negotiation successful
0809 12:16:25.005550 (+ 657us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":879,"thread_start_us":540,"threads_started":1}
I20260809 12:16:25.008527 21704 heartbeater.cc:347] Connected to a master server at 127.19.183.254:39447
I20260809 12:16:25.008885 21704 heartbeater.cc:464] Registering TS with master...
I20260809 12:16:25.009935 21704 heartbeater.cc:511] Master 127.19.183.254:39447 requested a full tablet report, sending...
I20260809 12:16:25.011435 21021 ts_manager.cc:194] Registered new tserver with Master: 23786868badc4d9a886244bd2b962b76 (127.19.183.195:36771)
I20260809 12:16:25.012485 21021 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:57517
I20260809 12:16:25.017016 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:16:25.043890 21707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:25.026061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56534 (local address 127.19.183.254:39447)
0809 12:16:25.026235 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:25.026240 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:25.027351 (+ 1111us) server_negotiation.cc:407] Connection header received
0809 12:16:25.027580 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:25.027585 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:25.027673 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:25.027781 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:25.030291 (+ 2510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.031192 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.032918 (+ 1726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.033503 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.033680 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:25.037071 (+ 3391us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:25.037076 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:25.037080 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:25.037111 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:25.040660 (+ 3549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:25.041386 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:25.041392 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:25.041396 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:25.041589 (+ 193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:25.042054 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:25.042061 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:25.042066 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:25.042502 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:25.042813 (+ 311us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:25.043282 (+ 469us) server_negotiation.cc:299] Negotiation successful
0809 12:16:25.043497 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:16:25.053380 21021 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56534:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:16:25.064497 21021 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:16:25.100459 21722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:25.084204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:39681 (local address 127.0.0.1:38234)
0809 12:16:25.085281 (+ 1077us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:25.085331 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:25.085698 (+ 367us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:25.088135 (+ 2437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:25.088143 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:25.088176 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:25.088705 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:25.088720 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.091509 (+ 2789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.091518 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:25.095310 (+ 3792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:25.095334 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.098933 (+ 3599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.098944 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:25.099290 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:25.099366 (+ 76us) client_negotiation.cc:769] Sending connection context
0809 12:16:25.099533 (+ 167us) client_negotiation.cc:240] Negotiation successful
0809 12:16:25.099815 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"thread_start_us":454,"threads_started":1}
I20260809 12:16:25.101734 21723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:25.084420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38234 (local address 127.19.183.193:39681)
0809 12:16:25.085499 (+ 1079us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:25.085508 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:25.085540 (+ 32us) server_negotiation.cc:407] Connection header received
0809 12:16:25.085967 (+ 427us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:25.085983 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:25.086272 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:25.086576 (+ 304us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:25.088974 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.091296 (+ 2322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.095635 (+ 4339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.098023 (+ 2388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.098424 (+ 401us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:25.098682 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:25.100954 (+ 2272us) server_negotiation.cc:299] Negotiation successful
0809 12:16:25.101220 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":853,"thread_start_us":600,"threads_started":1}
I20260809 12:16:25.107748 21224 tablet_service.cc:1467] Processing CreateTablet for tablet 0c04416083f6420687ca4ed093f9ab90 (DEFAULT_TABLE table=test-table [id=e6367e68cfcc4ce3b5551f5a8253433f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:25.109678 21224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c04416083f6420687ca4ed093f9ab90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:25.112164 21722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:25.089504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:36771 (local address 127.0.0.1:43300)
0809 12:16:25.102518 (+ 13014us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:25.102559 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:25.102665 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:25.104175 (+ 1510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:25.104182 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:25.104211 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:25.104893 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:25.104907 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.107182 (+ 2275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.107186 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:25.108876 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:25.108891 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.110871 (+ 1980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.110879 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:25.111134 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:25.111167 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 12:16:25.111273 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 12:16:25.111442 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12907,"mutex_wait_us":26,"thread_start_us":13580,"threads_started":1}
I20260809 12:16:25.112339 21729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:25.089748 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43300 (local address 127.19.183.195:36771)
0809 12:16:25.103643 (+ 13895us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:25.103649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:25.103683 (+ 34us) server_negotiation.cc:407] Connection header received
0809 12:16:25.103765 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:25.103769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:25.103910 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:25.104220 (+ 310us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:25.105131 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.106957 (+ 1826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.109082 (+ 2125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.110638 (+ 1556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.110898 (+ 260us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:25.111100 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:25.111370 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 12:16:25.111569 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13715,"thread_start_us":13357,"threads_started":1}
I20260809 12:16:25.113731 21726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:25.085758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45150 (local address 127.19.183.194:34439)
0809 12:16:25.101929 (+ 16171us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:25.101937 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:25.101985 (+ 48us) server_negotiation.cc:407] Connection header received
0809 12:16:25.102106 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:25.102114 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:25.102419 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:25.102692 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:25.104547 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.106445 (+ 1898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.109834 (+ 3389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.111622 (+ 1788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.112615 (+ 993us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:25.112841 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:25.112962 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 12:16:25.113187 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15942,"thread_start_us":15874,"threads_started":1}
I20260809 12:16:25.113888 21725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:25.085474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:34439 (local address 127.0.0.1:45150)
0809 12:16:25.098516 (+ 13042us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:25.098556 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:25.098783 (+ 227us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:25.102687 (+ 3904us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:25.102695 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:25.102753 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:25.104052 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:25.104073 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.106606 (+ 2533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.106611 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:25.109625 (+ 3014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:25.109644 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.112115 (+ 2471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.112136 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:25.112440 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:25.112489 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 12:16:25.112582 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 12:16:25.112758 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12868,"mutex_wait_us":70,"thread_start_us":12678,"threads_started":1}
I20260809 12:16:25.116984 21610 tablet_service.cc:1467] Processing CreateTablet for tablet 0c04416083f6420687ca4ed093f9ab90 (DEFAULT_TABLE table=test-table [id=e6367e68cfcc4ce3b5551f5a8253433f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:25.117437 21415 tablet_service.cc:1467] Processing CreateTablet for tablet 0c04416083f6420687ca4ed093f9ab90 (DEFAULT_TABLE table=test-table [id=e6367e68cfcc4ce3b5551f5a8253433f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:25.118808 21415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c04416083f6420687ca4ed093f9ab90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:25.119078 21610 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0c04416083f6420687ca4ed093f9ab90. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:25.132241 21730 tablet_bootstrap.cc:492] T 0c04416083f6420687ca4ed093f9ab90 P a43e59570f27491db98013483342326f: Bootstrap starting.
I20260809 12:16:25.134624 21731 tablet_bootstrap.cc:492] T 0c04416083f6420687ca4ed093f9ab90 P 4b6b40d09acd46e99f02d9f16f0a1635: Bootstrap starting.
I20260809 12:16:25.137492 21730 tablet_bootstrap.cc:654] T 0c04416083f6420687ca4ed093f9ab90 P a43e59570f27491db98013483342326f: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:25.138929 21732 tablet_bootstrap.cc:492] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76: Bootstrap starting.
I20260809 12:16:25.139596 21731 tablet_bootstrap.cc:654] T 0c04416083f6420687ca4ed093f9ab90 P 4b6b40d09acd46e99f02d9f16f0a1635: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:25.141027 21730 log.cc:824] T 0c04416083f6420687ca4ed093f9ab90 P a43e59570f27491db98013483342326f: Log is configured to *not* fsync() on all Append() calls
I20260809 12:16:25.141162 21731 log.cc:824] T 0c04416083f6420687ca4ed093f9ab90 P 4b6b40d09acd46e99f02d9f16f0a1635: Log is configured to *not* fsync() on all Append() calls
I20260809 12:16:25.144116 21732 tablet_bootstrap.cc:654] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:25.145121 21730 tablet_bootstrap.cc:492] T 0c04416083f6420687ca4ed093f9ab90 P a43e59570f27491db98013483342326f: No bootstrap required, opened a new log
I20260809 12:16:25.145377 21731 tablet_bootstrap.cc:492] T 0c04416083f6420687ca4ed093f9ab90 P 4b6b40d09acd46e99f02d9f16f0a1635: No bootstrap required, opened a new log
I20260809 12:16:25.145438 21730 ts_tablet_manager.cc:1397] T 0c04416083f6420687ca4ed093f9ab90 P a43e59570f27491db98013483342326f: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.010s
I20260809 12:16:25.145733 21731 ts_tablet_manager.cc:1397] T 0c04416083f6420687ca4ed093f9ab90 P 4b6b40d09acd46e99f02d9f16f0a1635: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.004s
I20260809 12:16:25.145810 21732 log.cc:824] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76: Log is configured to *not* fsync() on all Append() calls
I20260809 12:16:25.150102 21732 tablet_bootstrap.cc:492] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76: No bootstrap required, opened a new log
I20260809 12:16:25.150488 21732 ts_tablet_manager.cc:1397] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.007s
I20260809 12:16:25.156479 21730 raft_consensus.cc:348] T 0c04416083f6420687ca4ed093f9ab90 P a43e59570f27491db98013483342326f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a43e59570f27491db98013483342326f" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 39681 } } peers { permanent_uuid: "4b6b40d09acd46e99f02d9f16f0a1635" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 34439 } } peers { permanent_uuid: "23786868badc4d9a886244bd2b962b76" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36771 } }
I20260809 12:16:25.156987 21730 raft_consensus.cc:374] T 0c04416083f6420687ca4ed093f9ab90 P a43e59570f27491db98013483342326f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:25.157164 21730 raft_consensus.cc:729] T 0c04416083f6420687ca4ed093f9ab90 P a43e59570f27491db98013483342326f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a43e59570f27491db98013483342326f, State: Initialized, Role: FOLLOWER
I20260809 12:16:25.157982 21730 consensus_queue.cc:260] T 0c04416083f6420687ca4ed093f9ab90 P a43e59570f27491db98013483342326f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a43e59570f27491db98013483342326f" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 39681 } } peers { permanent_uuid: "4b6b40d09acd46e99f02d9f16f0a1635" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 34439 } } peers { permanent_uuid: "23786868badc4d9a886244bd2b962b76" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36771 } }
I20260809 12:16:25.159919 21731 raft_consensus.cc:348] T 0c04416083f6420687ca4ed093f9ab90 P 4b6b40d09acd46e99f02d9f16f0a1635 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a43e59570f27491db98013483342326f" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 39681 } } peers { permanent_uuid: "4b6b40d09acd46e99f02d9f16f0a1635" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 34439 } } peers { permanent_uuid: "23786868badc4d9a886244bd2b962b76" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36771 } }
I20260809 12:16:25.160480 21730 ts_tablet_manager.cc:1428] T 0c04416083f6420687ca4ed093f9ab90 P a43e59570f27491db98013483342326f: Time spent starting tablet: real 0.015s user 0.006s sys 0.007s
I20260809 12:16:25.160516 21731 raft_consensus.cc:374] T 0c04416083f6420687ca4ed093f9ab90 P 4b6b40d09acd46e99f02d9f16f0a1635 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:25.160633 21731 raft_consensus.cc:729] T 0c04416083f6420687ca4ed093f9ab90 P 4b6b40d09acd46e99f02d9f16f0a1635 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4b6b40d09acd46e99f02d9f16f0a1635, State: Initialized, Role: FOLLOWER
I20260809 12:16:25.161361 21731 consensus_queue.cc:260] T 0c04416083f6420687ca4ed093f9ab90 P 4b6b40d09acd46e99f02d9f16f0a1635 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a43e59570f27491db98013483342326f" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 39681 } } peers { permanent_uuid: "4b6b40d09acd46e99f02d9f16f0a1635" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 34439 } } peers { permanent_uuid: "23786868badc4d9a886244bd2b962b76" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36771 } }
I20260809 12:16:25.166934 21732 raft_consensus.cc:348] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a43e59570f27491db98013483342326f" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 39681 } } peers { permanent_uuid: "4b6b40d09acd46e99f02d9f16f0a1635" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 34439 } } peers { permanent_uuid: "23786868badc4d9a886244bd2b962b76" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36771 } }
I20260809 12:16:25.167346 21507 heartbeater.cc:503] Master 127.19.183.254:39447 was elected leader, sending a full tablet report...
I20260809 12:16:25.167614 21732 raft_consensus.cc:374] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:25.167775 21732 raft_consensus.cc:729] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23786868badc4d9a886244bd2b962b76, State: Initialized, Role: FOLLOWER
I20260809 12:16:25.168426 21732 consensus_queue.cc:260] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a43e59570f27491db98013483342326f" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 39681 } } peers { permanent_uuid: "4b6b40d09acd46e99f02d9f16f0a1635" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 34439 } } peers { permanent_uuid: "23786868badc4d9a886244bd2b962b76" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36771 } }
I20260809 12:16:25.169647 21731 ts_tablet_manager.cc:1428] T 0c04416083f6420687ca4ed093f9ab90 P 4b6b40d09acd46e99f02d9f16f0a1635: Time spent starting tablet: real 0.024s user 0.023s sys 0.000s
I20260809 12:16:25.170123 21704 heartbeater.cc:503] Master 127.19.183.254:39447 was elected leader, sending a full tablet report...
W20260809 12:16:25.170574 21510 tablet.cc:2369] T 0c04416083f6420687ca4ed093f9ab90 P 4b6b40d09acd46e99f02d9f16f0a1635: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:16:25.170675 21732 ts_tablet_manager.cc:1428] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76: Time spent starting tablet: real 0.020s user 0.006s sys 0.012s
W20260809 12:16:25.232167 21706 tablet.cc:2369] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:16:25.243446 21744 raft_consensus.cc:482] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:16:25.243865 21744 raft_consensus.cc:504] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a43e59570f27491db98013483342326f" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 39681 } } peers { permanent_uuid: "4b6b40d09acd46e99f02d9f16f0a1635" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 34439 } } peers { permanent_uuid: "23786868badc4d9a886244bd2b962b76" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36771 } }
I20260809 12:16:25.246088 21744 leader_election.cc:288] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a43e59570f27491db98013483342326f (127.19.183.193:39681), 4b6b40d09acd46e99f02d9f16f0a1635 (127.19.183.194:34439)
I20260809 12:16:25.254360 21726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:25.246933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:35535 (local address 127.19.183.194:34439)
0809 12:16:25.247251 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:25.247259 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:25.247279 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:16:25.247389 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:25.247395 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:25.247484 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:25.247588 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:25.248130 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.249568 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.252065 (+ 2497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.253448 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.253563 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:25.253618 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:25.254027 (+ 409us) server_negotiation.cc:299] Negotiation successful
0809 12:16:25.254133 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 12:16:25.254460 21708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:25.246375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:34439 (local address 127.19.183.195:35535)
0809 12:16:25.246675 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:25.246696 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:25.246884 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:25.247632 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:25.247636 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:25.247661 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:25.247949 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:25.247961 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.249710 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.249716 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:25.251825 (+ 2109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:25.251859 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.253608 (+ 1749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.253614 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:25.253858 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:25.253896 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 12:16:25.254000 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 12:16:25.254147 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"mutex_wait_us":1}
I20260809 12:16:25.255864 21723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:25.246419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:56559 (local address 127.19.183.193:39681)
0809 12:16:25.246739 (+ 320us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:25.246746 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:25.247780 (+ 1034us) server_negotiation.cc:407] Connection header received
0809 12:16:25.247963 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:25.247968 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:25.248054 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:25.248152 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:25.249308 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.250529 (+ 1221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.253245 (+ 2716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.254572 (+ 1327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.254797 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:25.254879 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:25.255423 (+ 544us) server_negotiation.cc:299] Negotiation successful
0809 12:16:25.255579 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260809 12:16:25.256170 21445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c04416083f6420687ca4ed093f9ab90" candidate_uuid: "23786868badc4d9a886244bd2b962b76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6b40d09acd46e99f02d9f16f0a1635" is_pre_election: true
I20260809 12:16:25.256700 21745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:25.246440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:39681 (local address 127.19.183.195:56559)
0809 12:16:25.247648 (+ 1208us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:25.247681 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:25.247824 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:25.248184 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:25.248189 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:25.248236 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:25.249142 (+ 906us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:25.249158 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.250690 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.250694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:25.253072 (+ 2378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:25.253091 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.254970 (+ 1879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.254975 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:25.255221 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:25.255255 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 12:16:25.256102 (+ 847us) client_negotiation.cc:240] Negotiation successful
0809 12:16:25.256243 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1055,"thread_start_us":754,"threads_started":1}
I20260809 12:16:25.257128 21445 raft_consensus.cc:2454] T 0c04416083f6420687ca4ed093f9ab90 P 4b6b40d09acd46e99f02d9f16f0a1635 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23786868badc4d9a886244bd2b962b76 in term 0.
I20260809 12:16:25.258181 21541 leader_election.cc:302] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23786868badc4d9a886244bd2b962b76, 4b6b40d09acd46e99f02d9f16f0a1635; no voters:
I20260809 12:16:25.257941 21258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c04416083f6420687ca4ed093f9ab90" candidate_uuid: "23786868badc4d9a886244bd2b962b76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a43e59570f27491db98013483342326f" is_pre_election: true
I20260809 12:16:25.258744 21744 raft_consensus.cc:2781] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:16:25.258765 21258 raft_consensus.cc:2454] T 0c04416083f6420687ca4ed093f9ab90 P a43e59570f27491db98013483342326f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23786868badc4d9a886244bd2b962b76 in term 0.
I20260809 12:16:25.258998 21744 raft_consensus.cc:482] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:16:25.259202 21744 raft_consensus.cc:3037] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:25.262141 21744 raft_consensus.cc:504] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a43e59570f27491db98013483342326f" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 39681 } } peers { permanent_uuid: "4b6b40d09acd46e99f02d9f16f0a1635" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 34439 } } peers { permanent_uuid: "23786868badc4d9a886244bd2b962b76" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36771 } }
I20260809 12:16:25.263028 21744 leader_election.cc:288] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76 [CANDIDATE]: Term 1 election: Requested vote from peers a43e59570f27491db98013483342326f (127.19.183.193:39681), 4b6b40d09acd46e99f02d9f16f0a1635 (127.19.183.194:34439)
I20260809 12:16:25.263381 21258 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c04416083f6420687ca4ed093f9ab90" candidate_uuid: "23786868badc4d9a886244bd2b962b76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a43e59570f27491db98013483342326f"
I20260809 12:16:25.263474 21445 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0c04416083f6420687ca4ed093f9ab90" candidate_uuid: "23786868badc4d9a886244bd2b962b76" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4b6b40d09acd46e99f02d9f16f0a1635"
I20260809 12:16:25.263613 21258 raft_consensus.cc:3037] T 0c04416083f6420687ca4ed093f9ab90 P a43e59570f27491db98013483342326f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:25.263741 21445 raft_consensus.cc:3037] T 0c04416083f6420687ca4ed093f9ab90 P 4b6b40d09acd46e99f02d9f16f0a1635 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:25.266489 21258 raft_consensus.cc:2454] T 0c04416083f6420687ca4ed093f9ab90 P a43e59570f27491db98013483342326f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23786868badc4d9a886244bd2b962b76 in term 1.
I20260809 12:16:25.266587 21445 raft_consensus.cc:2454] T 0c04416083f6420687ca4ed093f9ab90 P 4b6b40d09acd46e99f02d9f16f0a1635 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23786868badc4d9a886244bd2b962b76 in term 1.
I20260809 12:16:25.267024 21537 leader_election.cc:302] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23786868badc4d9a886244bd2b962b76, a43e59570f27491db98013483342326f; no voters:
I20260809 12:16:25.267369 21744 raft_consensus.cc:2781] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:16:25.268074 21744 raft_consensus.cc:686] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76 [term 1 LEADER]: Becoming Leader. State: Replica: 23786868badc4d9a886244bd2b962b76, State: Running, Role: LEADER
I20260809 12:16:25.268684 21744 consensus_queue.cc:237] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a43e59570f27491db98013483342326f" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 39681 } } peers { permanent_uuid: "4b6b40d09acd46e99f02d9f16f0a1635" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 34439 } } peers { permanent_uuid: "23786868badc4d9a886244bd2b962b76" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36771 } }
I20260809 12:16:25.274154 21021 catalog_manager.cc:5495] T 0c04416083f6420687ca4ed093f9ab90 P 23786868badc4d9a886244bd2b962b76 reported cstate change: term changed from 0 to 1, leader changed from <none> to 23786868badc4d9a886244bd2b962b76 (127.19.183.195). New cstate: current_term: 1 leader_uuid: "23786868badc4d9a886244bd2b962b76" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a43e59570f27491db98013483342326f" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 39681 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4b6b40d09acd46e99f02d9f16f0a1635" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 34439 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23786868badc4d9a886244bd2b962b76" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36771 } health_report { overall_health: HEALTHY } } }
Aug 09 12:16:25 dist-test-slave-39l8 krb5kdc[20962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277782, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.193@KRBTEST.COM
I20260809 12:16:25.315141 21723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:25.296890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38242 (local address 127.19.183.193:39681)
0809 12:16:25.297069 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:25.297074 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:25.297977 (+ 903us) server_negotiation.cc:407] Connection header received
0809 12:16:25.298235 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:25.298239 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:25.298308 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:25.298416 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:25.301124 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.302023 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.303666 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.304225 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.304442 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:25.309087 (+ 4645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:25.309102 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:25.309118 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:25.309165 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:25.312314 (+ 3149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:25.313085 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:25.313092 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:25.313095 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:25.313260 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:25.313617 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:25.313629 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:25.313631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:25.314024 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:25.314261 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:25.314563 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 12:16:25.314805 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Aug 09 12:16:25 dist-test-slave-39l8 krb5kdc[20962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277785, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 12:16:25 dist-test-slave-39l8 krb5kdc[20962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277785, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.183.193@KRBTEST.COM
W20260809 12:16:25.365321 21317 tablet.cc:2369] T 0c04416083f6420687ca4ed093f9ab90 P a43e59570f27491db98013483342326f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:16:25.369591 21723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:25.355105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38248 (local address 127.19.183.193:39681)
0809 12:16:25.355322 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:25.355331 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:25.355670 (+ 339us) server_negotiation.cc:407] Connection header received
0809 12:16:25.355883 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:25.355888 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:25.355954 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:25.356060 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:25.357394 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.358529 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.359886 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.360355 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.360495 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:25.364091 (+ 3596us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:25.364096 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:25.364100 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:25.364134 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:25.367007 (+ 2873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:25.367688 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:25.367695 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:25.367698 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:25.367882 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:25.368406 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:25.368409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:25.368411 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:25.368675 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:25.368869 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:25.369156 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 12:16:25.369312 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 12:16:25 dist-test-slave-39l8 krb5kdc[20962](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 12:16:25 dist-test-slave-39l8 krb5kdc[20962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.193@KRBTEST.COM
I20260809 12:16:25.420344 21723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:25.405527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38254 (local address 127.19.183.193:39681)
0809 12:16:25.405737 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:25.405746 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:25.406072 (+ 326us) server_negotiation.cc:407] Connection header received
0809 12:16:25.406305 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:25.406309 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:25.406375 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:25.406481 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:25.407883 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.408765 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.410032 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.410559 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.410724 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:25.414431 (+ 3707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:25.414435 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:25.414439 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:25.414471 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:25.417375 (+ 2904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:25.418105 (+ 730us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:25.418109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:25.418111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:25.418226 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:25.418786 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:25.418789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:25.418791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:25.419074 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:25.419362 (+ 288us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:25.419717 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 12:16:25.419954 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 12:16:25.422523 21187 tablet_service.cc:2908] Rejecting scan request for tablet 0c04416083f6420687ca4ed093f9ab90: Uninitialized: clean time has not yet been initialized
I20260809 12:16:25.426878 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 21115
I20260809 12:16:25.439915 21723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:25.427780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38266 (local address 127.19.183.193:39681)
0809 12:16:25.427978 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:25.427986 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:25.428330 (+ 344us) server_negotiation.cc:407] Connection header received
0809 12:16:25.428482 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:25.428486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:25.428554 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:25.428636 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:25.430141 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.430874 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.432667 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.433052 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.433154 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:25.435357 (+ 2203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:25.435362 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:25.435365 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:25.435394 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:25.437633 (+ 2239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:25.438195 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:25.438198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:25.438200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:25.438305 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:25.438624 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:25.438627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:25.438629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:25.438964 (+ 335us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:25.439187 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:25.439420 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 12:16:25.439585 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 12:16:25.440587 21309 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:16:25.529831 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 21115
I20260809 12:16:25.562168 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 21323
Aug 09 12:16:25 dist-test-slave-39l8 krb5kdc[20962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.194@KRBTEST.COM
I20260809 12:16:25.576601 21726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:25.563141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45152 (local address 127.19.183.194:34439)
0809 12:16:25.563301 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:25.563305 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:25.563322 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:16:25.563381 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:25.563384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:25.563447 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:25.563551 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:25.564716 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.565474 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.566661 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.567159 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.567254 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:25.571324 (+ 4070us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:25.571339 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:25.571354 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:25.571385 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:25.574188 (+ 2803us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:25.574827 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:25.574834 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:25.574836 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:25.574958 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:25.575284 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:25.575288 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:25.575289 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:25.575657 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:25.575906 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:25.576071 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 12:16:25.576239 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 12:16:25.577297 21499 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:16:25.666590 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 21323
I20260809 12:16:25.694460 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 21516
Aug 09 12:16:25 dist-test-slave-39l8 krb5kdc[20962](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277785, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.195@KRBTEST.COM
I20260809 12:16:25.709174 21762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:25.695334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43312 (local address 127.19.183.195:36771)
0809 12:16:25.695825 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:25.695830 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:25.695847 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:16:25.695904 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:25.695908 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:25.695974 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:25.696067 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:25.697438 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.698176 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.699501 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:25.699874 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:25.699991 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:25.704140 (+ 4149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:25.704154 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:25.704171 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:25.704202 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:25.706904 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:25.707495 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:25.707502 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:25.707504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:25.707653 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:25.707978 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:25.707981 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:25.707983 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:25.708287 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:25.708506 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:25.708706 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 12:16:25.708870 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":267,"threads_started":1}
I20260809 12:16:25.709865 21698 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:16:25.797953 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 21516
I20260809 12:16:25.823280 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 20980
I20260809 12:16:25.824505 21066 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:16:25.931777 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 20980
2026-08-09T12:16:25Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (4672 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:16:26 dist-test-slave-39l8 krb5kdc[21768](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:16:26 dist-test-slave-39l8 krb5kdc[21768](info): set up 2 sockets
Aug 09 12:16:26 dist-test-slave-39l8 krb5kdc[21768](info): commencing operation
krb5kdc: starting...
W20260809 12:16:27.227869 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.003s sys 0.018s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:16:27 dist-test-slave-39l8 krb5kdc[21768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:16:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:16:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:16:27.430049 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:33911
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:36057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:33911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:27.762024 21788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:27.762315 21788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:27.762507 21788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:27.769723 21788 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:16:27.769855 21788 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 12:16:27.769896 21788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:27.769982 21788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:27.770084 21788 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:16:27.770190 21788 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:16:27.780146 21788 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:36057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:33911
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:33911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:27.781450 21788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:27.782817 21788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:27.794507 21796 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:27.794548 21793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:27.794543 21794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:27.795434 21788 server_base.cc:1034] running on GCE node
I20260809 12:16:27.796243 21788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:27.797675 21788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:27.798906 21788 hybrid_clock.cc:648] HybridClock initialized: now 1786277787798858 us; error 44 us; skew 500 ppm
I20260809 12:16:27.799242 21788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:27 dist-test-slave-39l8 krb5kdc[21768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277787, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:27.802443 21788 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:16:27.804113 21788 webserver.cc:460] Webserver started at http://127.19.183.254:37271/ using document root <none> and password file <none>
I20260809 12:16:27.804817 21788 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:27.804955 21788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:27.805290 21788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:27.808352 21788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "c70e29d6c6f742c781abe41e95fc5fcc"
format_stamp: "Formatted at 2026-08-09 12:16:27 on dist-test-slave-39l8"
server_key: "1c0647ea4a60a9cabc4ff9d905edd45a"
server_key_iv: "6f1de2136cca32c9c49e6df2f5f75e93"
server_key_version: "encryptionkey@0"
I20260809 12:16:27.809149 21788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "c70e29d6c6f742c781abe41e95fc5fcc"
format_stamp: "Formatted at 2026-08-09 12:16:27 on dist-test-slave-39l8"
server_key: "1c0647ea4a60a9cabc4ff9d905edd45a"
server_key_iv: "6f1de2136cca32c9c49e6df2f5f75e93"
server_key_version: "encryptionkey@0"
I20260809 12:16:27.814132 21788 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 12:16:27.817615 21807 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:27.818851 21788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:16:27.819028 21788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "c70e29d6c6f742c781abe41e95fc5fcc"
format_stamp: "Formatted at 2026-08-09 12:16:27 on dist-test-slave-39l8"
server_key: "1c0647ea4a60a9cabc4ff9d905edd45a"
server_key_iv: "6f1de2136cca32c9c49e6df2f5f75e93"
server_key_version: "encryptionkey@0"
I20260809 12:16:27.819252 21788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:27.841001 21788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:27.844900 21788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:27.845186 21788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:27.866576 21788 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:33911
I20260809 12:16:27.866613 21894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:33911 every 8 connection(s)
I20260809 12:16:27.867985 21788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:16:27.871678 21895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:27.878268 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 21788
I20260809 12:16:27.878490 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:16:27.879199 20191 external_mini_cluster.cc:1428] Setting key 362c6dc0604a83e09665d3f32fc7fe70
I20260809 12:16:27.882076 21895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc: Bootstrap starting.
I20260809 12:16:27.886179 21895 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:27.887372 21895 log.cc:824] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc: Log is configured to *not* fsync() on all Append() calls
I20260809 12:16:27.890275 21895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc: No bootstrap required, opened a new log
Aug 09 12:16:27 dist-test-slave-39l8 krb5kdc[21768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277787, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:27.898681 21895 raft_consensus.cc:348] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70e29d6c6f742c781abe41e95fc5fcc" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33911 } }
I20260809 12:16:27.899143 21895 raft_consensus.cc:374] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:27.899291 21895 raft_consensus.cc:729] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c70e29d6c6f742c781abe41e95fc5fcc, State: Initialized, Role: FOLLOWER
I20260809 12:16:27.899915 21895 consensus_queue.cc:260] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70e29d6c6f742c781abe41e95fc5fcc" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33911 } }
I20260809 12:16:27.900189 21895 raft_consensus.cc:388] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:16:27.900338 21895 raft_consensus.cc:482] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:16:27.900535 21895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:27.903152 21895 raft_consensus.cc:504] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70e29d6c6f742c781abe41e95fc5fcc" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33911 } }
I20260809 12:16:27.903525 21898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:27.881186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35748 (local address 127.19.183.254:33911)
0809 12:16:27.882149 (+ 963us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:27.882164 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:27.882218 (+ 54us) server_negotiation.cc:407] Connection header received
0809 12:16:27.884780 (+ 2562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:27.884808 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:27.885294 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:27.885737 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:27.887219 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:27.889081 (+ 1862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:27.890370 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:27.891456 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:27.891654 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:27.895988 (+ 4334us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:27.895996 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:27.896010 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:27.896058 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:27.899279 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:27.899977 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:27.899986 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:27.899993 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:27.900209 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:27.900597 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:27.900602 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:27.900605 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:27.901148 (+ 543us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:27.901471 (+ 323us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:27.901943 (+ 472us) server_negotiation.cc:299] Negotiation successful
0809 12:16:27.902407 (+ 464us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":458,"threads_started":1}
I20260809 12:16:27.903738 21895 leader_election.cc:302] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c70e29d6c6f742c781abe41e95fc5fcc; no voters:
I20260809 12:16:27.904397 21895 leader_election.cc:288] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:16:27.904554 21900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:16:27.906355 21895 sys_catalog.cc:564] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:16:27.907397 21900 raft_consensus.cc:686] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc [term 1 LEADER]: Becoming Leader. State: Replica: c70e29d6c6f742c781abe41e95fc5fcc, State: Running, Role: LEADER
I20260809 12:16:27.907979 21900 consensus_queue.cc:237] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70e29d6c6f742c781abe41e95fc5fcc" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33911 } }
I20260809 12:16:27.910866 21903 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc [sys.catalog]: SysCatalogTable state changed. Reason: New leader c70e29d6c6f742c781abe41e95fc5fcc. Latest consensus state: current_term: 1 leader_uuid: "c70e29d6c6f742c781abe41e95fc5fcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70e29d6c6f742c781abe41e95fc5fcc" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33911 } } }
I20260809 12:16:27.911424 21901 sys_catalog.cc:455] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c70e29d6c6f742c781abe41e95fc5fcc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c70e29d6c6f742c781abe41e95fc5fcc" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33911 } } }
I20260809 12:16:27.911605 21903 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc [sys.catalog]: This master's current role is: LEADER
I20260809 12:16:27.912299 21901 sys_catalog.cc:458] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc [sys.catalog]: This master's current role is: LEADER
I20260809 12:16:27.912536 21912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:16:27.918072 21912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:16:27.929005 21912 catalog_manager.cc:1347] Generated new cluster ID: ab96aad98d984f1087b200b925f59667
I20260809 12:16:27.929140 21912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:16:27.956485 21912 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:16:27.958173 21912 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:16:27.965838 21912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc: Generated new TSK 0
I20260809 12:16:27.966671 21912 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
I20260809 12:16:28.022396 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:33911
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:36057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:28.357872 21933 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:28.358202 21933 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:28.358369 21933 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:28.365268 21933 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:28.365417 21933 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:28.365567 21933 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:16:28.365718 21933 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:16:28.375059 21933 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:36057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.183.254:33911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:28.376322 21933 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:28.377713 21933 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:28.391733 21940 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:28.391731 21942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:28.391733 21939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:28.392590 21933 server_base.cc:1034] running on GCE node
I20260809 12:16:28.393282 21933 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:28.394306 21933 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:28.395553 21933 hybrid_clock.cc:648] HybridClock initialized: now 1786277788395487 us; error 41 us; skew 500 ppm
I20260809 12:16:28.395869 21933 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:28 dist-test-slave-39l8 krb5kdc[21768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277788, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:28.399132 21933 init.cc:377] Logged in from keytab as kudu/127.19.183.193@KRBTEST.COM (short username kudu)
I20260809 12:16:28.400467 21933 webserver.cc:460] Webserver started at http://127.19.183.193:32991/ using document root <none> and password file <none>
I20260809 12:16:28.401350 21933 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:28.402297 21933 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:28.402681 21933 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:28.405905 21933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2"
format_stamp: "Formatted at 2026-08-09 12:16:28 on dist-test-slave-39l8"
server_key: "cc1eee18d826861a438a714ebe6ea762"
server_key_iv: "fe80e7bec812f6c1b8cecd6a77e7b8bb"
server_key_version: "encryptionkey@0"
I20260809 12:16:28.406791 21933 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2"
format_stamp: "Formatted at 2026-08-09 12:16:28 on dist-test-slave-39l8"
server_key: "cc1eee18d826861a438a714ebe6ea762"
server_key_iv: "fe80e7bec812f6c1b8cecd6a77e7b8bb"
server_key_version: "encryptionkey@0"
I20260809 12:16:28.411754 21933 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 12:16:28.415089 21949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:28.416347 21933 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:16:28.416476 21933 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2"
format_stamp: "Formatted at 2026-08-09 12:16:28 on dist-test-slave-39l8"
server_key: "cc1eee18d826861a438a714ebe6ea762"
server_key_iv: "fe80e7bec812f6c1b8cecd6a77e7b8bb"
server_key_version: "encryptionkey@0"
I20260809 12:16:28.416664 21933 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:28.433372 21933 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:28.437311 21933 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:28.437539 21933 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:28.440418 21933 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:28.440541 21933 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:28.440773 21933 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:28.440882 21933 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:16:28 dist-test-slave-39l8 krb5kdc[21768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277788, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:28.464581 21962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:28.440792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:56167 (local address 127.19.183.254:33911)
0809 12:16:28.441258 (+ 466us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:28.441267 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:28.442444 (+ 1177us) server_negotiation.cc:407] Connection header received
0809 12:16:28.444673 (+ 2229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:28.444681 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:28.444782 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:28.444914 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:28.447157 (+ 2243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:28.448270 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:28.451113 (+ 2843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:28.451768 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:28.451911 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:28.457231 (+ 5320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:28.457236 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:28.457239 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:28.457274 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:28.460364 (+ 3090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:28.461373 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:28.461380 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:28.461383 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:28.461564 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:28.462201 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:28.462208 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:28.462214 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:28.462667 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:28.462861 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:28.463975 (+ 1114us) server_negotiation.cc:299] Negotiation successful
0809 12:16:28.464202 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":240,"threads_started":1}
I20260809 12:16:28.466128 21965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:28.441506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:33911 (local address 127.19.183.193:56167)
0809 12:16:28.442287 (+ 781us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:28.442327 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:28.444294 (+ 1967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:28.445143 (+ 849us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:28.445158 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:28.445967 (+ 809us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:28.446927 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:28.446960 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:28.448500 (+ 1540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:28.448508 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:28.450912 (+ 2404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:28.450928 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:28.452058 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:28.452068 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:28.452280 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:28.453105 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:28.453148 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:28.456965 (+ 3817us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:28.460546 (+ 3581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:28.460557 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:28.460580 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:28.461186 (+ 606us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:28.461677 (+ 491us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:28.461683 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:28.461687 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:28.461991 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:28.462954 (+ 963us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:28.462962 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:28.463505 (+ 543us) client_negotiation.cc:769] Sending connection context
0809 12:16:28.463853 (+ 348us) client_negotiation.cc:240] Negotiation successful
0809 12:16:28.464471 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":543,"thread_start_us":331,"threads_started":1}
I20260809 12:16:28.488960 21933 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:36301
I20260809 12:16:28.489001 22113 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:36301 every 8 connection(s)
I20260809 12:16:28.489893 22074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:28.474018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:33911 (local address 127.19.183.193:54395)
0809 12:16:28.475180 (+ 1162us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:28.475199 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:28.475361 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:28.475905 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:28.475908 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:28.476635 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:28.477137 (+ 502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:28.477158 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:28.478263 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:28.478268 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:28.480076 (+ 1808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:28.480089 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:28.480867 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:28.480873 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:28.481044 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:28.481682 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:28.481700 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:28.482439 (+ 739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:28.486669 (+ 4230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:28.486679 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:28.486684 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:28.487156 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:28.487646 (+ 490us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:28.487654 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:28.487658 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:28.487839 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:28.488432 (+ 593us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:28.488440 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:28.488689 (+ 249us) client_negotiation.cc:769] Sending connection context
0809 12:16:28.488930 (+ 241us) client_negotiation.cc:240] Negotiation successful
0809 12:16:28.489320 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1031,"thread_start_us":894,"threads_started":1}
I20260809 12:16:28.490459 21933 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:16:28.490694 21962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:28.474395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:54395 (local address 127.19.183.254:33911)
0809 12:16:28.474570 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:28.474574 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:28.475289 (+ 715us) server_negotiation.cc:407] Connection header received
0809 12:16:28.475496 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:28.475503 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:28.475601 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:28.475716 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:28.477363 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:28.478112 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:28.480248 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:28.480670 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:28.480814 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:28.482659 (+ 1845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:28.482665 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:28.482668 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:28.482698 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:28.486444 (+ 3746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:28.487281 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:28.487287 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:28.487302 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:28.487478 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:28.487997 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:28.488004 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:28.488010 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:28.488295 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:28.488474 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:28.490242 (+ 1768us) server_negotiation.cc:299] Negotiation successful
0809 12:16:28.490407 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:16:28.492677 22116 heartbeater.cc:347] Connected to a master server at 127.19.183.254:33911
I20260809 12:16:28.493042 22116 heartbeater.cc:464] Registering TS with master...
I20260809 12:16:28.493906 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 21933
I20260809 12:16:28.493939 22116 heartbeater.cc:511] Master 127.19.183.254:33911 requested a full tablet report, sending...
I20260809 12:16:28.494171 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:16:28.495047 20191 external_mini_cluster.cc:1428] Setting key e634c432f20cac3069a05b6494448d48
I20260809 12:16:28.496309 21833 ts_manager.cc:194] Registered new tserver with Master: b107dcbd8c3e4b6a9951ff6b7494c4a2 (127.19.183.193:36301)
I20260809 12:16:28.498275 21833 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:56167
WARNING: no policy specified for kudu/127.19.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
I20260809 12:16:28.539520 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:33911
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:36057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:28.861222 22122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:28.861515 22122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:28.861682 22122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:28.870397 22122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:28.870525 22122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:28.870656 22122 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:16:28.870791 22122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:16:28.885746 22122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:36057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.183.254:33911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:28.887123 22122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:28.888708 22122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:28.901530 22132 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:28.901615 22135 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:28.901603 22131 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:28.902269 22122 server_base.cc:1034] running on GCE node
I20260809 12:16:28.902894 22122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:28.903780 22122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:28.905033 22122 hybrid_clock.cc:648] HybridClock initialized: now 1786277788904960 us; error 47 us; skew 500 ppm
I20260809 12:16:28.905313 22122 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:28 dist-test-slave-39l8 krb5kdc[21768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277788, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:28.912603 22122 init.cc:377] Logged in from keytab as kudu/127.19.183.194@KRBTEST.COM (short username kudu)
I20260809 12:16:28.913969 22122 webserver.cc:460] Webserver started at http://127.19.183.194:45027/ using document root <none> and password file <none>
I20260809 12:16:28.914670 22122 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:28.914839 22122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:28.915190 22122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:28.918001 22122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a"
format_stamp: "Formatted at 2026-08-09 12:16:28 on dist-test-slave-39l8"
server_key: "a98fca5c6372fc7b027cd614e78ee01b"
server_key_iv: "d5f9a4fd65351d27af63cd00a42142cc"
server_key_version: "encryptionkey@0"
I20260809 12:16:28.918860 22122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a"
format_stamp: "Formatted at 2026-08-09 12:16:28 on dist-test-slave-39l8"
server_key: "a98fca5c6372fc7b027cd614e78ee01b"
server_key_iv: "d5f9a4fd65351d27af63cd00a42142cc"
server_key_version: "encryptionkey@0"
I20260809 12:16:28.924364 22122 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:16:28.927722 22144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:28.928985 22122 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:16:28.929158 22122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a"
format_stamp: "Formatted at 2026-08-09 12:16:28 on dist-test-slave-39l8"
server_key: "a98fca5c6372fc7b027cd614e78ee01b"
server_key_iv: "d5f9a4fd65351d27af63cd00a42142cc"
server_key_version: "encryptionkey@0"
I20260809 12:16:28.929360 22122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:28.948854 22122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:28.953122 22122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:28.953409 22122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:28.956079 22122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:28.956192 22122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:28.956370 22122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:28.956493 22122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 09 12:16:28 dist-test-slave-39l8 krb5kdc[21768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277788, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:28.978833 21962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:28.956610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:36623 (local address 127.19.183.254:33911)
0809 12:16:28.956797 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:28.956804 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:28.958074 (+ 1270us) server_negotiation.cc:407] Connection header received
0809 12:16:28.960143 (+ 2069us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:28.960151 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:28.960247 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:28.960364 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:28.961750 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:28.963054 (+ 1304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:28.965574 (+ 2520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:28.966371 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:28.966583 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:28.971682 (+ 5099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:28.971690 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:28.971696 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:28.971743 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:28.975014 (+ 3271us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:28.975905 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:28.975913 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:28.975919 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:28.976076 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:28.976903 (+ 827us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:28.976910 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:28.976917 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:28.977367 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:28.977552 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:28.978228 (+ 676us) server_negotiation.cc:299] Negotiation successful
0809 12:16:28.978460 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 12:16:28.980067 22153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:28.957088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:33911 (local address 127.19.183.194:36623)
0809 12:16:28.957885 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:28.957936 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:28.959814 (+ 1878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:28.960556 (+ 742us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:28.960563 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:28.961023 (+ 460us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:28.961552 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:28.961568 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:28.963247 (+ 1679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:28.963252 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:28.965394 (+ 2142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:28.965416 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:28.966761 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:28.966772 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:28.967098 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:28.967928 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:28.968078 (+ 150us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:28.971435 (+ 3357us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:28.975209 (+ 3774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:28.975233 (+ 24us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:28.975253 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:28.975776 (+ 523us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:28.976230 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:28.976241 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:28.976246 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:28.976731 (+ 485us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:28.977526 (+ 795us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:28.977534 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:28.977851 (+ 317us) client_negotiation.cc:769] Sending connection context
0809 12:16:28.978129 (+ 278us) client_negotiation.cc:240] Negotiation successful
0809 12:16:28.978596 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":372,"threads_started":1}
I20260809 12:16:29.009023 22122 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:41741
I20260809 12:16:29.009133 22308 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:41741 every 8 connection(s)
I20260809 12:16:29.010697 22122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:16:29.012710 22251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:28.993375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:33911 (local address 127.19.183.194:53419)
0809 12:16:28.995441 (+ 2066us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:28.995462 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:28.995681 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:28.996194 (+ 513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:28.996203 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:28.997065 (+ 862us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:28.997711 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:28.997729 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:28.999406 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:28.999413 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:29.000884 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:29.000903 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.002349 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.002358 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:29.002602 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:29.003427 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:29.003450 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:29.004991 (+ 1541us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:29.009415 (+ 4424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:29.009419 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:29.009423 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:29.009944 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:29.010338 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:29.010342 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:29.010345 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:29.010486 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:29.011152 (+ 666us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:29.011160 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:29.011432 (+ 272us) client_negotiation.cc:769] Sending connection context
0809 12:16:29.011668 (+ 236us) client_negotiation.cc:240] Negotiation successful
0809 12:16:29.012319 (+ 651us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1885,"thread_start_us":1692,"threads_started":1}
I20260809 12:16:29.013193 22309 heartbeater.cc:347] Connected to a master server at 127.19.183.254:33911
I20260809 12:16:29.013422 22309 heartbeater.cc:464] Registering TS with master...
I20260809 12:16:29.013520 21962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:28.993261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:53419 (local address 127.19.183.254:33911)
0809 12:16:28.993411 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:28.993419 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:28.995563 (+ 2144us) server_negotiation.cc:407] Connection header received
0809 12:16:28.995886 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:28.995893 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:28.995989 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:28.996112 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:28.997935 (+ 1823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:28.999234 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.001068 (+ 1834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.001896 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.002043 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:29.005237 (+ 3194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:29.005246 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:29.005252 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:29.005317 (+ 65us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:29.009240 (+ 3923us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:29.010086 (+ 846us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:29.010089 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:29.010092 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:29.010220 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:29.010594 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:29.010597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:29.010599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:29.011014 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:29.011131 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:29.013031 (+ 1900us) server_negotiation.cc:299] Negotiation successful
0809 12:16:29.013206 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 12:16:29.014201 22309 heartbeater.cc:511] Master 127.19.183.254:33911 requested a full tablet report, sending...
I20260809 12:16:29.015722 21833 ts_manager.cc:194] Registered new tserver with Master: ae8d3b2563e64040bd6b49e50f3b2c9a (127.19.183.194:41741)
I20260809 12:16:29.016523 21833 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:36623
I20260809 12:16:29.017349 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 22122
I20260809 12:16:29.017552 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:16:29.017992 20191 external_mini_cluster.cc:1428] Setting key 83a5e0764958d6512856fc3ecda4ca31
WARNING: no policy specified for kudu/127.19.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
I20260809 12:16:29.063414 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:33911
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:36057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:29.388260 22314 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:29.388561 22314 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:29.388728 22314 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:29.396035 22314 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:29.396183 22314 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:29.396322 22314 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:16:29.396456 22314 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:16:29.405984 22314 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:36057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.183.254:33911
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:29.407313 22314 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:29.408675 22314 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:29.422299 22327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:29.422326 22325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:29.422259 22324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:29.423352 22314 server_base.cc:1034] running on GCE node
I20260809 12:16:29.423930 22314 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:29.424921 22314 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:29.426134 22314 hybrid_clock.cc:648] HybridClock initialized: now 1786277789426077 us; error 32 us; skew 500 ppm
I20260809 12:16:29.426417 22314 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:29 dist-test-slave-39l8 krb5kdc[21768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277789, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:29.429566 22314 init.cc:377] Logged in from keytab as kudu/127.19.183.195@KRBTEST.COM (short username kudu)
I20260809 12:16:29.431097 22314 webserver.cc:460] Webserver started at http://127.19.183.195:43557/ using document root <none> and password file <none>
I20260809 12:16:29.431737 22314 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:29.431824 22314 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:29.432120 22314 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:29.435138 22314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "f2f8a12891e24319b23d664cd5cf4917"
format_stamp: "Formatted at 2026-08-09 12:16:29 on dist-test-slave-39l8"
server_key: "59a1912dcf9ce11737bf1e01c24dc069"
server_key_iv: "08d29eff44a98690856a60543c671a10"
server_key_version: "encryptionkey@0"
I20260809 12:16:29.435902 22314 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "f2f8a12891e24319b23d664cd5cf4917"
format_stamp: "Formatted at 2026-08-09 12:16:29 on dist-test-slave-39l8"
server_key: "59a1912dcf9ce11737bf1e01c24dc069"
server_key_iv: "08d29eff44a98690856a60543c671a10"
server_key_version: "encryptionkey@0"
I20260809 12:16:29.440796 22314 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:16:29.444046 22340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:29.445308 22314 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 12:16:29.445494 22314 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "f2f8a12891e24319b23d664cd5cf4917"
format_stamp: "Formatted at 2026-08-09 12:16:29 on dist-test-slave-39l8"
server_key: "59a1912dcf9ce11737bf1e01c24dc069"
server_key_iv: "08d29eff44a98690856a60543c671a10"
server_key_version: "encryptionkey@0"
I20260809 12:16:29.445719 22314 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:29.462996 22314 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:29.467054 22314 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:29.467353 22314 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:29.470032 22314 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:29.470146 22314 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:29.470306 22314 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:29.470381 22314 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:16:29 dist-test-slave-39l8 krb5kdc[21768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277789, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:29.495308 21962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:29.470844 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:56583 (local address 127.19.183.254:33911)
0809 12:16:29.471036 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:29.471043 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:29.472510 (+ 1467us) server_negotiation.cc:407] Connection header received
0809 12:16:29.474571 (+ 2061us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:29.474576 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:29.474682 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:29.474818 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:29.477104 (+ 2286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.478465 (+ 1361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.481820 (+ 3355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.482434 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.482596 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:29.487465 (+ 4869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:29.487471 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:29.487476 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:29.487518 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:29.491112 (+ 3594us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:29.492295 (+ 1183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:29.492300 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:29.492304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:29.492467 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:29.493052 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:29.493057 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:29.493061 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:29.493428 (+ 367us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:29.493595 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:29.494592 (+ 997us) server_negotiation.cc:299] Negotiation successful
0809 12:16:29.494880 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 12:16:29.496874 22352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:29.471403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:33911 (local address 127.19.183.195:56583)
0809 12:16:29.472308 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:29.472356 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:29.474289 (+ 1933us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:29.475086 (+ 797us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:29.475101 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:29.475934 (+ 833us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:29.476792 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:29.476816 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.478746 (+ 1930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.478756 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:29.481668 (+ 2912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:29.481683 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.483022 (+ 1339us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.483032 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:29.483306 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:29.484148 (+ 842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:29.484190 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:29.487256 (+ 3066us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:29.491299 (+ 4043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:29.491325 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:29.491346 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:29.492111 (+ 765us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:29.492599 (+ 488us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:29.492608 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:29.492614 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:29.492907 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:29.493583 (+ 676us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:29.493595 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:29.494099 (+ 504us) client_negotiation.cc:769] Sending connection context
0809 12:16:29.494436 (+ 337us) client_negotiation.cc:240] Negotiation successful
0809 12:16:29.495077 (+ 641us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":633,"thread_start_us":396,"threads_started":1}
I20260809 12:16:29.500926 22116 heartbeater.cc:503] Master 127.19.183.254:33911 was elected leader, sending a full tablet report...
I20260809 12:16:29.521041 21962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:29.505975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:52253 (local address 127.19.183.254:33911)
0809 12:16:29.506172 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:29.506176 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:29.507106 (+ 930us) server_negotiation.cc:407] Connection header received
0809 12:16:29.507374 (+ 268us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:29.507383 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:29.507467 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:29.507604 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:29.509370 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.510599 (+ 1229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.512620 (+ 2021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.513237 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.513391 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:29.515385 (+ 1994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:29.515390 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:29.515393 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:29.515422 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:29.518129 (+ 2707us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:29.518830 (+ 701us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:29.518842 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:29.518844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:29.518974 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:29.519380 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:29.519384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:29.519386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:29.519650 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:29.519871 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:29.520407 (+ 536us) server_negotiation.cc:299] Negotiation successful
0809 12:16:29.520712 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 12:16:29.521093 22423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:29.506118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:33911 (local address 127.19.183.195:52253)
0809 12:16:29.506981 (+ 863us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:29.507007 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:29.507185 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:29.507648 (+ 463us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:29.507656 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:29.508558 (+ 902us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:29.509184 (+ 626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:29.509203 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.510763 (+ 1560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.510771 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:29.512423 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:29.512439 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.513439 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.513446 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:29.513617 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:29.514367 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:29.514384 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:29.515197 (+ 813us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:29.518263 (+ 3066us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:29.518269 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:29.518273 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:29.518670 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:29.519097 (+ 427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:29.519102 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:29.519106 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:29.519236 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:29.519932 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:29.519936 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:29.520156 (+ 220us) client_negotiation.cc:769] Sending connection context
0809 12:16:29.520296 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 12:16:29.520614 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":665,"thread_start_us":408,"threads_started":1}
I20260809 12:16:29.537695 22314 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:33047
I20260809 12:16:29.537727 22502 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:33047 every 8 connection(s)
I20260809 12:16:29.539287 22314 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:16:29.541527 22503 heartbeater.cc:347] Connected to a master server at 127.19.183.254:33911
I20260809 12:16:29.541712 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 22314
I20260809 12:16:29.541764 22503 heartbeater.cc:464] Registering TS with master...
I20260809 12:16:29.541898 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:16:29.542325 20191 external_mini_cluster.cc:1428] Setting key 738bbb07e5b6cb3d1d95342be867ea43
I20260809 12:16:29.542552 22503 heartbeater.cc:511] Master 127.19.183.254:33911 requested a full tablet report, sending...
I20260809 12:16:29.544256 21833 ts_manager.cc:194] Registered new tserver with Master: f2f8a12891e24319b23d664cd5cf4917 (127.19.183.195:33047)
I20260809 12:16:29.545337 21833 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:56583
I20260809 12:16:29.545593 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 12:16:29 dist-test-slave-39l8 krb5kdc[21768](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277789, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 12:16:29 dist-test-slave-39l8 krb5kdc[21768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277789, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:29.596128 21962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:29.582938 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35758 (local address 127.19.183.254:33911)
0809 12:16:29.583187 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:29.583195 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:29.583369 (+ 174us) server_negotiation.cc:407] Connection header received
0809 12:16:29.583586 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:29.583596 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:29.583661 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:29.583767 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:29.585108 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.585912 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.587312 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.587708 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.587809 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:29.591366 (+ 3557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:29.591370 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:29.591374 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:29.591403 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:29.593822 (+ 2419us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:29.594375 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:29.594379 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:29.594381 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:29.594497 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:29.594877 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:29.594880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:29.594882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:29.595157 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:29.595338 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:29.595564 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 12:16:29.595725 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 12:16:29.600081 21833 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35758:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:16:29.607093 21833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:16:29.632565 22519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:29.620032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:33047 (local address 127.0.0.1:58876)
0809 12:16:29.620737 (+ 705us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:29.620751 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:29.620840 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:29.622995 (+ 2155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:29.623002 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:29.623041 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:29.623590 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:29.623609 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.626343 (+ 2734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.626351 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:29.629332 (+ 2981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:29.629353 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.631469 (+ 2116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.631478 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:29.631775 (+ 297us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:29.631850 (+ 75us) client_negotiation.cc:769] Sending connection context
0809 12:16:29.631971 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 12:16:29.632208 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":593,"thread_start_us":983,"threads_started":1}
I20260809 12:16:29.633852 22520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:29.620948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58876 (local address 127.19.183.195:33047)
0809 12:16:29.621807 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:29.621815 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:29.621849 (+ 34us) server_negotiation.cc:407] Connection header received
0809 12:16:29.622475 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:29.622491 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:29.622756 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:29.623869 (+ 1113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:29.624240 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.626189 (+ 1949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.629600 (+ 3411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.631192 (+ 1592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.633119 (+ 1927us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:29.633286 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:29.633365 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 12:16:29.633530 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":685,"thread_start_us":480,"threads_started":1}
I20260809 12:16:29.637501 22407 tablet_service.cc:1467] Processing CreateTablet for tablet a2863d9f0dfe4b679358a569a49cba2e (DEFAULT_TABLE table=test-table [id=ca6affae3c3b43cbba85bf47cc4cc405]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:29.639065 22407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2863d9f0dfe4b679358a569a49cba2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:29.640086 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:29.619550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56188 (local address 127.19.183.193:36301)
0809 12:16:29.632482 (+ 12932us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:29.632491 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:29.632518 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:16:29.632640 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:29.632648 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:29.632959 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:29.633152 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:29.633781 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.635345 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.637636 (+ 2291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.638994 (+ 1358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.639212 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:29.639370 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:29.639627 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 12:16:29.639780 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12784,"thread_start_us":12625,"threads_started":1}
I20260809 12:16:29.640116 22517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:29.619500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:36301 (local address 127.0.0.1:56188)
0809 12:16:29.620505 (+ 1005us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:29.620549 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:29.620816 (+ 267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:29.633184 (+ 12368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:29.633192 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:29.633230 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:29.633628 (+ 398us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:29.633644 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.635474 (+ 1830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.635480 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:29.637441 (+ 1961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:29.637458 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.639251 (+ 1793us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.639258 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:29.639419 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:29.639454 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 12:16:29.639561 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 12:16:29.639704 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":689,"mutex_wait_us":23,"thread_start_us":2441,"threads_started":1}
I20260809 12:16:29.641120 22521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:29.621676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:41741 (local address 127.0.0.1:59282)
0809 12:16:29.632292 (+ 10616us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:29.632311 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:29.632431 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:29.634274 (+ 1843us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:29.634279 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:29.634304 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:29.634658 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:29.634673 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.636523 (+ 1850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.636528 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:29.638250 (+ 1722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:29.638264 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.640050 (+ 1786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.640063 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:29.640569 (+ 506us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:29.640599 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 12:16:29.640695 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 12:16:29.640801 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10484,"mutex_wait_us":1,"thread_start_us":10268,"threads_started":1}
I20260809 12:16:29.642172 22522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:29.622178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59282 (local address 127.19.183.194:41741)
0809 12:16:29.633716 (+ 11538us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:29.633724 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:29.633748 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:16:29.633845 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:29.633853 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:29.634077 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:29.634285 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:29.634885 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.636369 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.638402 (+ 2033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.639564 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.639732 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:29.639896 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:29.641698 (+ 1802us) server_negotiation.cc:299] Negotiation successful
0809 12:16:29.641878 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11369,"thread_start_us":11247,"threads_started":1}
I20260809 12:16:29.643863 22022 tablet_service.cc:1467] Processing CreateTablet for tablet a2863d9f0dfe4b679358a569a49cba2e (DEFAULT_TABLE table=test-table [id=ca6affae3c3b43cbba85bf47cc4cc405]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:29.645254 22022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2863d9f0dfe4b679358a569a49cba2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:29.645556 22211 tablet_service.cc:1467] Processing CreateTablet for tablet a2863d9f0dfe4b679358a569a49cba2e (DEFAULT_TABLE table=test-table [id=ca6affae3c3b43cbba85bf47cc4cc405]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:29.646896 22211 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2863d9f0dfe4b679358a569a49cba2e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:29.650888 22523 tablet_bootstrap.cc:492] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917: Bootstrap starting.
I20260809 12:16:29.653724 22523 tablet_bootstrap.cc:654] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:29.654753 22523 log.cc:824] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917: Log is configured to *not* fsync() on all Append() calls
I20260809 12:16:29.659847 22525 tablet_bootstrap.cc:492] T a2863d9f0dfe4b679358a569a49cba2e P b107dcbd8c3e4b6a9951ff6b7494c4a2: Bootstrap starting.
I20260809 12:16:29.661298 22527 tablet_bootstrap.cc:492] T a2863d9f0dfe4b679358a569a49cba2e P ae8d3b2563e64040bd6b49e50f3b2c9a: Bootstrap starting.
I20260809 12:16:29.662832 22523 tablet_bootstrap.cc:492] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917: No bootstrap required, opened a new log
I20260809 12:16:29.663132 22523 ts_tablet_manager.cc:1397] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917: Time spent bootstrapping tablet: real 0.012s user 0.005s sys 0.004s
I20260809 12:16:29.663942 22525 tablet_bootstrap.cc:654] T a2863d9f0dfe4b679358a569a49cba2e P b107dcbd8c3e4b6a9951ff6b7494c4a2: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:29.665088 22527 tablet_bootstrap.cc:654] T a2863d9f0dfe4b679358a569a49cba2e P ae8d3b2563e64040bd6b49e50f3b2c9a: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:29.665706 22525 log.cc:824] T a2863d9f0dfe4b679358a569a49cba2e P b107dcbd8c3e4b6a9951ff6b7494c4a2: Log is configured to *not* fsync() on all Append() calls
I20260809 12:16:29.666623 22527 log.cc:824] T a2863d9f0dfe4b679358a569a49cba2e P ae8d3b2563e64040bd6b49e50f3b2c9a: Log is configured to *not* fsync() on all Append() calls
I20260809 12:16:29.673796 22527 tablet_bootstrap.cc:492] T a2863d9f0dfe4b679358a569a49cba2e P ae8d3b2563e64040bd6b49e50f3b2c9a: No bootstrap required, opened a new log
I20260809 12:16:29.673827 22525 tablet_bootstrap.cc:492] T a2863d9f0dfe4b679358a569a49cba2e P b107dcbd8c3e4b6a9951ff6b7494c4a2: No bootstrap required, opened a new log
I20260809 12:16:29.674140 22525 ts_tablet_manager.cc:1397] T a2863d9f0dfe4b679358a569a49cba2e P b107dcbd8c3e4b6a9951ff6b7494c4a2: Time spent bootstrapping tablet: real 0.014s user 0.007s sys 0.004s
I20260809 12:16:29.674154 22527 ts_tablet_manager.cc:1397] T a2863d9f0dfe4b679358a569a49cba2e P ae8d3b2563e64040bd6b49e50f3b2c9a: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.008s
I20260809 12:16:29.675964 22523 raft_consensus.cc:348] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } }
I20260809 12:16:29.676693 22523 raft_consensus.cc:374] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:29.676890 22523 raft_consensus.cc:729] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2f8a12891e24319b23d664cd5cf4917, State: Initialized, Role: FOLLOWER
I20260809 12:16:29.677954 22523 consensus_queue.cc:260] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } }
I20260809 12:16:29.679313 22503 heartbeater.cc:503] Master 127.19.183.254:33911 was elected leader, sending a full tablet report...
I20260809 12:16:29.680326 22523 ts_tablet_manager.cc:1428] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917: Time spent starting tablet: real 0.017s user 0.013s sys 0.003s
I20260809 12:16:29.688828 22527 raft_consensus.cc:348] T a2863d9f0dfe4b679358a569a49cba2e P ae8d3b2563e64040bd6b49e50f3b2c9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } }
I20260809 12:16:29.689383 22527 raft_consensus.cc:374] T a2863d9f0dfe4b679358a569a49cba2e P ae8d3b2563e64040bd6b49e50f3b2c9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:29.689545 22527 raft_consensus.cc:729] T a2863d9f0dfe4b679358a569a49cba2e P ae8d3b2563e64040bd6b49e50f3b2c9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae8d3b2563e64040bd6b49e50f3b2c9a, State: Initialized, Role: FOLLOWER
I20260809 12:16:29.690425 22527 consensus_queue.cc:260] T a2863d9f0dfe4b679358a569a49cba2e P ae8d3b2563e64040bd6b49e50f3b2c9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } }
I20260809 12:16:29.690651 22525 raft_consensus.cc:348] T a2863d9f0dfe4b679358a569a49cba2e P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } }
I20260809 12:16:29.691318 22525 raft_consensus.cc:374] T a2863d9f0dfe4b679358a569a49cba2e P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:29.691526 22525 raft_consensus.cc:729] T a2863d9f0dfe4b679358a569a49cba2e P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b107dcbd8c3e4b6a9951ff6b7494c4a2, State: Initialized, Role: FOLLOWER
I20260809 12:16:29.692304 22525 consensus_queue.cc:260] T a2863d9f0dfe4b679358a569a49cba2e P b107dcbd8c3e4b6a9951ff6b7494c4a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } }
I20260809 12:16:29.699925 22309 heartbeater.cc:503] Master 127.19.183.254:33911 was elected leader, sending a full tablet report...
I20260809 12:16:29.700667 22527 ts_tablet_manager.cc:1428] T a2863d9f0dfe4b679358a569a49cba2e P ae8d3b2563e64040bd6b49e50f3b2c9a: Time spent starting tablet: real 0.026s user 0.017s sys 0.006s
I20260809 12:16:29.701107 22525 ts_tablet_manager.cc:1428] T a2863d9f0dfe4b679358a569a49cba2e P b107dcbd8c3e4b6a9951ff6b7494c4a2: Time spent starting tablet: real 0.027s user 0.012s sys 0.011s
I20260809 12:16:29.738375 22530 raft_consensus.cc:482] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:16:29.739354 22530 raft_consensus.cc:504] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } }
I20260809 12:16:29.742475 22530 leader_election.cc:288] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b107dcbd8c3e4b6a9951ff6b7494c4a2 (127.19.183.193:36301), ae8d3b2563e64040bd6b49e50f3b2c9a (127.19.183.194:41741)
W20260809 12:16:29.744352 22117 tablet.cc:2369] T a2863d9f0dfe4b679358a569a49cba2e P b107dcbd8c3e4b6a9951ff6b7494c4a2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:16:29.749563 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:29.742261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:54453 (local address 127.19.183.193:36301)
0809 12:16:29.742507 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:29.742513 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:29.742532 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:16:29.743241 (+ 709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:29.743246 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:29.743329 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:29.743426 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:29.743991 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.745080 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.747370 (+ 2290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.748386 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.748590 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:29.748666 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:29.749119 (+ 453us) server_negotiation.cc:299] Negotiation successful
0809 12:16:29.749241 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 12:16:29.749583 22352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:29.741988 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:36301 (local address 127.19.183.195:54453)
0809 12:16:29.742390 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:29.742410 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:29.742543 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:29.743487 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:29.743494 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:29.743521 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:29.743835 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:29.743846 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.745314 (+ 1468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.745318 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:29.747182 (+ 1864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:29.747196 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.748555 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.748577 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:29.748861 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:29.748891 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 12:16:29.749046 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 12:16:29.749179 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":2}
I20260809 12:16:29.750880 22047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2863d9f0dfe4b679358a569a49cba2e" candidate_uuid: "f2f8a12891e24319b23d664cd5cf4917" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" is_pre_election: true
I20260809 12:16:29.751703 22047 raft_consensus.cc:2454] T a2863d9f0dfe4b679358a569a49cba2e P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2f8a12891e24319b23d664cd5cf4917 in term 0.
I20260809 12:16:29.751842 22522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:29.743167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:36237 (local address 127.19.183.194:41741)
0809 12:16:29.743388 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:29.743395 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:29.743801 (+ 406us) server_negotiation.cc:407] Connection header received
0809 12:16:29.744091 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:29.744096 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:29.744182 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:29.744275 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:29.745865 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.746690 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.749144 (+ 2454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.750689 (+ 1545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.750928 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:29.751009 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:29.751303 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 12:16:29.751443 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260809 12:16:29.752033 22533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:29.742054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:41741 (local address 127.19.183.195:36237)
0809 12:16:29.743649 (+ 1595us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:29.743665 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:29.743895 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:29.744355 (+ 460us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:29.744361 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:29.744399 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:29.745581 (+ 1182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:29.745600 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.746919 (+ 1319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.746924 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:29.748907 (+ 1983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:29.748919 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.750986 (+ 2067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.750993 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:29.751147 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:29.751169 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 12:16:29.751275 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 12:16:29.751412 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1405,"mutex_wait_us":28,"thread_start_us":1103,"threads_started":1}
I20260809 12:16:29.752671 22346 leader_election.cc:302] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b107dcbd8c3e4b6a9951ff6b7494c4a2, f2f8a12891e24319b23d664cd5cf4917; no voters:
I20260809 12:16:29.753207 22530 raft_consensus.cc:2781] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:16:29.753433 22530 raft_consensus.cc:482] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:16:29.753319 22244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2863d9f0dfe4b679358a569a49cba2e" candidate_uuid: "f2f8a12891e24319b23d664cd5cf4917" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" is_pre_election: true
I20260809 12:16:29.753645 22530 raft_consensus.cc:3037] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:29.753885 22244 raft_consensus.cc:2454] T a2863d9f0dfe4b679358a569a49cba2e P ae8d3b2563e64040bd6b49e50f3b2c9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2f8a12891e24319b23d664cd5cf4917 in term 0.
I20260809 12:16:29.757220 22530 raft_consensus.cc:504] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } }
I20260809 12:16:29.758055 22530 leader_election.cc:288] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917 [CANDIDATE]: Term 1 election: Requested vote from peers b107dcbd8c3e4b6a9951ff6b7494c4a2 (127.19.183.193:36301), ae8d3b2563e64040bd6b49e50f3b2c9a (127.19.183.194:41741)
I20260809 12:16:29.758442 22047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2863d9f0dfe4b679358a569a49cba2e" candidate_uuid: "f2f8a12891e24319b23d664cd5cf4917" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2"
I20260809 12:16:29.758662 22047 raft_consensus.cc:3037] T a2863d9f0dfe4b679358a569a49cba2e P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:29.758656 22244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2863d9f0dfe4b679358a569a49cba2e" candidate_uuid: "f2f8a12891e24319b23d664cd5cf4917" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a"
I20260809 12:16:29.758960 22244 raft_consensus.cc:3037] T a2863d9f0dfe4b679358a569a49cba2e P ae8d3b2563e64040bd6b49e50f3b2c9a [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:29.761747 22047 raft_consensus.cc:2454] T a2863d9f0dfe4b679358a569a49cba2e P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2f8a12891e24319b23d664cd5cf4917 in term 1.
I20260809 12:16:29.762243 22244 raft_consensus.cc:2454] T a2863d9f0dfe4b679358a569a49cba2e P ae8d3b2563e64040bd6b49e50f3b2c9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2f8a12891e24319b23d664cd5cf4917 in term 1.
I20260809 12:16:29.762290 22346 leader_election.cc:302] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b107dcbd8c3e4b6a9951ff6b7494c4a2, f2f8a12891e24319b23d664cd5cf4917; no voters:
I20260809 12:16:29.762761 22530 raft_consensus.cc:2781] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917 [term 1 FOLLOWER]: Leader election won for term 1
W20260809 12:16:29.762987 22311 tablet.cc:2369] T a2863d9f0dfe4b679358a569a49cba2e P ae8d3b2563e64040bd6b49e50f3b2c9a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:16:29.764029 22530 raft_consensus.cc:686] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917 [term 1 LEADER]: Becoming Leader. State: Replica: f2f8a12891e24319b23d664cd5cf4917, State: Running, Role: LEADER
I20260809 12:16:29.764743 22530 consensus_queue.cc:237] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } }
I20260809 12:16:29.771186 21832 catalog_manager.cc:5495] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2f8a12891e24319b23d664cd5cf4917 (127.19.183.195). New cstate: current_term: 1 leader_uuid: "f2f8a12891e24319b23d664cd5cf4917" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } health_report { overall_health: UNKNOWN } } }
W20260809 12:16:29.791136 22504 tablet.cc:2369] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:16:29.836010 22520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:29.828949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58890 (local address 127.19.183.195:33047)
0809 12:16:29.829240 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:29.829245 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:29.829264 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:16:29.829516 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:29.829521 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:29.829637 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:29.829767 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:16:29.830240 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.831364 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.833115 (+ 1751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:29.833836 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:29.833954 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:29.834652 (+ 698us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:16:29.835293 (+ 641us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:16:29.835402 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:29.835552 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 12:16:29.835619 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 12:16:29.847234 22244 raft_consensus.cc:1261] T a2863d9f0dfe4b679358a569a49cba2e P ae8d3b2563e64040bd6b49e50f3b2c9a [term 1 FOLLOWER]: Refusing update from remote peer f2f8a12891e24319b23d664cd5cf4917: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:16:29.848621 22534 consensus_queue.cc:1035] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 12:16:29.849874 22047 raft_consensus.cc:1261] T a2863d9f0dfe4b679358a569a49cba2e P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 1 FOLLOWER]: Refusing update from remote peer f2f8a12891e24319b23d664cd5cf4917: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:16:29.851004 22534 consensus_queue.cc:1035] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:16:29.904011 21833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:16:29.904619 21833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:16:29.910610 21833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc: Sending DeleteTablet for 3 replicas of tablet a2863d9f0dfe4b679358a569a49cba2e
I20260809 12:16:29.912426 22407 tablet_service.cc:1514] Processing DeleteTablet for tablet a2863d9f0dfe4b679358a569a49cba2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:16:29 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:58876
I20260809 12:16:29.912405 22022 tablet_service.cc:1514] Processing DeleteTablet for tablet a2863d9f0dfe4b679358a569a49cba2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:16:29 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:56188
I20260809 12:16:29.913148 22211 tablet_service.cc:1514] Processing DeleteTablet for tablet a2863d9f0dfe4b679358a569a49cba2e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:16:29 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:59282
I20260809 12:16:29.918138 21832 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35758:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:16:29.919165 21832 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:16:29.924366 22554 tablet_replica.cc:330] T a2863d9f0dfe4b679358a569a49cba2e P b107dcbd8c3e4b6a9951ff6b7494c4a2: stopping tablet replica
I20260809 12:16:29.925225 22554 raft_consensus.cc:2229] T a2863d9f0dfe4b679358a569a49cba2e P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:16:29.925822 22551 tablet_replica.cc:330] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917: stopping tablet replica
I20260809 12:16:29.926133 22554 raft_consensus.cc:2258] T a2863d9f0dfe4b679358a569a49cba2e P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:16:29.926616 22551 raft_consensus.cc:2229] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:16:29.928109 22551 raft_consensus.cc:2258] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:16:29.928752 22022 tablet_service.cc:1467] Processing CreateTablet for tablet c1155315e70047cd8bd796d04b977895 (DEFAULT_TABLE table=test-table [id=677d3870f286425ca517eda84a0cdfc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:29.929385 22022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1155315e70047cd8bd796d04b977895. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:29.929628 22407 tablet_service.cc:1467] Processing CreateTablet for tablet c1155315e70047cd8bd796d04b977895 (DEFAULT_TABLE table=test-table [id=677d3870f286425ca517eda84a0cdfc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:29.930078 22407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1155315e70047cd8bd796d04b977895. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:29.931335 22551 ts_tablet_manager.cc:1905] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:16:29.936553 22554 ts_tablet_manager.cc:1905] T a2863d9f0dfe4b679358a569a49cba2e P b107dcbd8c3e4b6a9951ff6b7494c4a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:16:29.940316 22208 tablet_service.cc:1467] Processing CreateTablet for tablet c1155315e70047cd8bd796d04b977895 (DEFAULT_TABLE table=test-table [id=677d3870f286425ca517eda84a0cdfc0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:29.941083 22523 tablet_bootstrap.cc:492] T c1155315e70047cd8bd796d04b977895 P f2f8a12891e24319b23d664cd5cf4917: Bootstrap starting.
I20260809 12:16:29.941440 22208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1155315e70047cd8bd796d04b977895. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:29.946169 22552 tablet_replica.cc:330] T a2863d9f0dfe4b679358a569a49cba2e P ae8d3b2563e64040bd6b49e50f3b2c9a: stopping tablet replica
I20260809 12:16:29.946805 22552 raft_consensus.cc:2229] T a2863d9f0dfe4b679358a569a49cba2e P ae8d3b2563e64040bd6b49e50f3b2c9a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:16:29.947449 22523 tablet_bootstrap.cc:654] T c1155315e70047cd8bd796d04b977895 P f2f8a12891e24319b23d664cd5cf4917: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:29.948202 22525 tablet_bootstrap.cc:492] T c1155315e70047cd8bd796d04b977895 P b107dcbd8c3e4b6a9951ff6b7494c4a2: Bootstrap starting.
I20260809 12:16:29.948202 22552 raft_consensus.cc:2258] T a2863d9f0dfe4b679358a569a49cba2e P ae8d3b2563e64040bd6b49e50f3b2c9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:16:29.950610 22554 ts_tablet_manager.cc:1918] T a2863d9f0dfe4b679358a569a49cba2e P b107dcbd8c3e4b6a9951ff6b7494c4a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:16:29.950954 22554 log.cc:1196] T a2863d9f0dfe4b679358a569a49cba2e P b107dcbd8c3e4b6a9951ff6b7494c4a2: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/a2863d9f0dfe4b679358a569a49cba2e
I20260809 12:16:29.951726 22525 tablet_bootstrap.cc:654] T c1155315e70047cd8bd796d04b977895 P b107dcbd8c3e4b6a9951ff6b7494c4a2: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:29.954218 22527 tablet_bootstrap.cc:492] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a: Bootstrap starting.
I20260809 12:16:29.954377 22552 ts_tablet_manager.cc:1905] T a2863d9f0dfe4b679358a569a49cba2e P ae8d3b2563e64040bd6b49e50f3b2c9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:16:29.954456 22554 ts_tablet_manager.cc:1939] T a2863d9f0dfe4b679358a569a49cba2e P b107dcbd8c3e4b6a9951ff6b7494c4a2: Deleting consensus metadata
I20260809 12:16:29.957373 22527 tablet_bootstrap.cc:654] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:29.958016 21813 catalog_manager.cc:4841] TS b107dcbd8c3e4b6a9951ff6b7494c4a2 (127.19.183.193:36301): tablet a2863d9f0dfe4b679358a569a49cba2e (table test-table [id=ca6affae3c3b43cbba85bf47cc4cc405]) successfully deleted
I20260809 12:16:29.958248 22523 tablet_bootstrap.cc:492] T c1155315e70047cd8bd796d04b977895 P f2f8a12891e24319b23d664cd5cf4917: No bootstrap required, opened a new log
I20260809 12:16:29.958514 22523 ts_tablet_manager.cc:1397] T c1155315e70047cd8bd796d04b977895 P f2f8a12891e24319b23d664cd5cf4917: Time spent bootstrapping tablet: real 0.018s user 0.002s sys 0.003s
I20260809 12:16:29.959507 22551 ts_tablet_manager.cc:1918] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:16:29.959807 22551 log.cc:1196] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/a2863d9f0dfe4b679358a569a49cba2e
I20260809 12:16:29.960547 22523 raft_consensus.cc:348] T c1155315e70047cd8bd796d04b977895 P f2f8a12891e24319b23d664cd5cf4917 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } }
I20260809 12:16:29.960845 22551 ts_tablet_manager.cc:1939] T a2863d9f0dfe4b679358a569a49cba2e P f2f8a12891e24319b23d664cd5cf4917: Deleting consensus metadata
I20260809 12:16:29.960933 22523 raft_consensus.cc:374] T c1155315e70047cd8bd796d04b977895 P f2f8a12891e24319b23d664cd5cf4917 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:29.961170 22523 raft_consensus.cc:729] T c1155315e70047cd8bd796d04b977895 P f2f8a12891e24319b23d664cd5cf4917 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2f8a12891e24319b23d664cd5cf4917, State: Initialized, Role: FOLLOWER
I20260809 12:16:29.961647 22523 consensus_queue.cc:260] T c1155315e70047cd8bd796d04b977895 P f2f8a12891e24319b23d664cd5cf4917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } }
I20260809 12:16:29.964546 22525 tablet_bootstrap.cc:492] T c1155315e70047cd8bd796d04b977895 P b107dcbd8c3e4b6a9951ff6b7494c4a2: No bootstrap required, opened a new log
I20260809 12:16:29.964781 22525 ts_tablet_manager.cc:1397] T c1155315e70047cd8bd796d04b977895 P b107dcbd8c3e4b6a9951ff6b7494c4a2: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.010s
I20260809 12:16:29.965811 22523 ts_tablet_manager.cc:1428] T c1155315e70047cd8bd796d04b977895 P f2f8a12891e24319b23d664cd5cf4917: Time spent starting tablet: real 0.007s user 0.004s sys 0.000s
I20260809 12:16:29.966238 22525 raft_consensus.cc:348] T c1155315e70047cd8bd796d04b977895 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } }
I20260809 12:16:29.966555 22525 raft_consensus.cc:374] T c1155315e70047cd8bd796d04b977895 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:29.966830 22525 raft_consensus.cc:729] T c1155315e70047cd8bd796d04b977895 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b107dcbd8c3e4b6a9951ff6b7494c4a2, State: Initialized, Role: FOLLOWER
I20260809 12:16:29.967478 22525 consensus_queue.cc:260] T c1155315e70047cd8bd796d04b977895 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } }
I20260809 12:16:29.969849 22525 ts_tablet_manager.cc:1428] T c1155315e70047cd8bd796d04b977895 P b107dcbd8c3e4b6a9951ff6b7494c4a2: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 12:16:29.972005 21811 catalog_manager.cc:4841] TS f2f8a12891e24319b23d664cd5cf4917 (127.19.183.195:33047): tablet a2863d9f0dfe4b679358a569a49cba2e (table test-table [id=ca6affae3c3b43cbba85bf47cc4cc405]) successfully deleted
I20260809 12:16:29.975509 22552 ts_tablet_manager.cc:1918] T a2863d9f0dfe4b679358a569a49cba2e P ae8d3b2563e64040bd6b49e50f3b2c9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:16:29.975826 22552 log.cc:1196] T a2863d9f0dfe4b679358a569a49cba2e P ae8d3b2563e64040bd6b49e50f3b2c9a: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/a2863d9f0dfe4b679358a569a49cba2e
I20260809 12:16:29.976974 22552 ts_tablet_manager.cc:1939] T a2863d9f0dfe4b679358a569a49cba2e P ae8d3b2563e64040bd6b49e50f3b2c9a: Deleting consensus metadata
I20260809 12:16:29.978034 22527 tablet_bootstrap.cc:492] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a: No bootstrap required, opened a new log
I20260809 12:16:29.978332 22527 ts_tablet_manager.cc:1397] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a: Time spent bootstrapping tablet: real 0.024s user 0.002s sys 0.010s
I20260809 12:16:29.979449 21811 catalog_manager.cc:4841] TS ae8d3b2563e64040bd6b49e50f3b2c9a (127.19.183.194:41741): tablet a2863d9f0dfe4b679358a569a49cba2e (table test-table [id=ca6affae3c3b43cbba85bf47cc4cc405]) successfully deleted
I20260809 12:16:29.980067 22527 raft_consensus.cc:348] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } }
I20260809 12:16:29.980419 22527 raft_consensus.cc:374] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:29.980698 22527 raft_consensus.cc:729] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae8d3b2563e64040bd6b49e50f3b2c9a, State: Initialized, Role: FOLLOWER
I20260809 12:16:29.981186 22527 consensus_queue.cc:260] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } }
I20260809 12:16:29.982301 22527 ts_tablet_manager.cc:1428] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260809 12:16:30.089792 22531 raft_consensus.cc:482] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:16:30.090090 22531 raft_consensus.cc:504] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } }
I20260809 12:16:30.092070 22531 leader_election.cc:288] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b107dcbd8c3e4b6a9951ff6b7494c4a2 (127.19.183.193:36301), f2f8a12891e24319b23d664cd5cf4917 (127.19.183.195:33047)
I20260809 12:16:30.103621 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:30.092768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:46925 (local address 127.19.183.193:36301)
0809 12:16:30.093067 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:30.093074 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:30.094172 (+ 1098us) server_negotiation.cc:407] Connection header received
0809 12:16:30.094544 (+ 372us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:30.094553 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:30.094671 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:30.094788 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:30.096116 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.097459 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.100614 (+ 3155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.102197 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.102387 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:30.102471 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:30.103052 (+ 581us) server_negotiation.cc:299] Negotiation successful
0809 12:16:30.103196 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260809 12:16:30.104106 22560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:30.092260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:36301 (local address 127.19.183.194:46925)
0809 12:16:30.094035 (+ 1775us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:30.094060 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:30.094339 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:30.095077 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:30.095085 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:30.095121 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:30.095880 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:30.095898 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.097609 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.097614 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:30.100433 (+ 2819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:30.100450 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.102503 (+ 2053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.102510 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:30.102781 (+ 271us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:30.102848 (+ 67us) client_negotiation.cc:769] Sending connection context
0809 12:16:30.102984 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 12:16:30.103117 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1566,"thread_start_us":4395,"threads_started":1}
I20260809 12:16:30.105136 22047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1155315e70047cd8bd796d04b977895" candidate_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" is_pre_election: true
I20260809 12:16:30.105461 22047 raft_consensus.cc:2454] T c1155315e70047cd8bd796d04b977895 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae8d3b2563e64040bd6b49e50f3b2c9a in term 0.
I20260809 12:16:30.106050 22148 leader_election.cc:302] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae8d3b2563e64040bd6b49e50f3b2c9a, b107dcbd8c3e4b6a9951ff6b7494c4a2; no voters:
I20260809 12:16:30.106348 22520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:30.093489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:48547 (local address 127.19.183.195:33047)
0809 12:16:30.093686 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:30.093693 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:30.097042 (+ 3349us) server_negotiation.cc:407] Connection header received
0809 12:16:30.097227 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:30.097230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:30.097307 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:30.097411 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:30.098455 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.099832 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.102192 (+ 2360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.105011 (+ 2819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.105192 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:30.105278 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:30.105745 (+ 467us) server_negotiation.cc:299] Negotiation successful
0809 12:16:30.105914 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260809 12:16:30.106585 22531 raft_consensus.cc:2781] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:16:30.106879 22531 raft_consensus.cc:482] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:16:30.107103 22531 raft_consensus.cc:3037] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:30.107303 22559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:30.092511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:33047 (local address 127.19.183.194:48547)
0809 12:16:30.096910 (+ 4399us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:30.096925 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:30.097081 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:30.097433 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:30.097438 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:30.097462 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:30.098219 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:30.098234 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.099993 (+ 1759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.100008 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:30.101975 (+ 1967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:30.101987 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.105303 (+ 3316us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.105309 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:30.105522 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:30.105560 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 12:16:30.106525 (+ 965us) client_negotiation.cc:240] Negotiation successful
0809 12:16:30.106687 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4279,"thread_start_us":1177,"threads_started":1}
I20260809 12:16:30.108292 22436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1155315e70047cd8bd796d04b977895" candidate_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2f8a12891e24319b23d664cd5cf4917" is_pre_election: true
I20260809 12:16:30.108731 22436 raft_consensus.cc:2454] T c1155315e70047cd8bd796d04b977895 P f2f8a12891e24319b23d664cd5cf4917 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ae8d3b2563e64040bd6b49e50f3b2c9a in term 0.
I20260809 12:16:30.111150 22531 raft_consensus.cc:504] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } }
I20260809 12:16:30.112026 22531 leader_election.cc:288] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a [CANDIDATE]: Term 1 election: Requested vote from peers b107dcbd8c3e4b6a9951ff6b7494c4a2 (127.19.183.193:36301), f2f8a12891e24319b23d664cd5cf4917 (127.19.183.195:33047)
I20260809 12:16:30.112527 22047 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1155315e70047cd8bd796d04b977895" candidate_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2"
I20260809 12:16:30.112753 22047 raft_consensus.cc:3037] T c1155315e70047cd8bd796d04b977895 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:30.112915 22436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1155315e70047cd8bd796d04b977895" candidate_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2f8a12891e24319b23d664cd5cf4917"
I20260809 12:16:30.113135 22436 raft_consensus.cc:3037] T c1155315e70047cd8bd796d04b977895 P f2f8a12891e24319b23d664cd5cf4917 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:30.116442 22047 raft_consensus.cc:2454] T c1155315e70047cd8bd796d04b977895 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae8d3b2563e64040bd6b49e50f3b2c9a in term 1.
I20260809 12:16:30.116964 22148 leader_election.cc:302] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ae8d3b2563e64040bd6b49e50f3b2c9a, b107dcbd8c3e4b6a9951ff6b7494c4a2; no voters:
I20260809 12:16:30.117170 22436 raft_consensus.cc:2454] T c1155315e70047cd8bd796d04b977895 P f2f8a12891e24319b23d664cd5cf4917 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ae8d3b2563e64040bd6b49e50f3b2c9a in term 1.
I20260809 12:16:30.117324 22531 raft_consensus.cc:2781] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:16:30.117998 22531 raft_consensus.cc:686] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a [term 1 LEADER]: Becoming Leader. State: Replica: ae8d3b2563e64040bd6b49e50f3b2c9a, State: Running, Role: LEADER
I20260809 12:16:30.118584 22531 consensus_queue.cc:237] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } }
I20260809 12:16:30.123545 21833 catalog_manager.cc:5495] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a reported cstate change: term changed from 0 to 1, leader changed from <none> to ae8d3b2563e64040bd6b49e50f3b2c9a (127.19.183.194). New cstate: current_term: 1 leader_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } health_report { overall_health: UNKNOWN } } }
Aug 09 12:16:30 dist-test-slave-39l8 krb5kdc[21768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277787, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:30.174320 22568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:30.156892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:33911 (local address 127.0.0.1:35762)
0809 12:16:30.157636 (+ 744us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:30.157658 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:30.157808 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:30.158404 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:30.158410 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:30.159398 (+ 988us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:30.159747 (+ 349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:30.159761 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.161105 (+ 1344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.161111 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:30.162840 (+ 1729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:30.162856 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.163671 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.163679 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:30.163862 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:30.164601 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:30.164626 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:30.167575 (+ 2949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:30.171508 (+ 3933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:30.171523 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:30.171531 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:30.171962 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:30.172434 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:30.172439 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:30.172443 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:30.172594 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:30.173136 (+ 542us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:30.173145 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:30.173321 (+ 176us) client_negotiation.cc:769] Sending connection context
0809 12:16:30.173483 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 12:16:30.173908 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":125,"thread_start_us":401,"threads_started":1}
I20260809 12:16:30.174320 22569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:30.156989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35762 (local address 127.19.183.254:33911)
0809 12:16:30.158072 (+ 1083us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:30.158077 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:30.158096 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:16:30.158146 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:30.158149 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:30.158204 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:30.158295 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:30.159917 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.160943 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.163009 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.163480 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.163575 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:30.167807 (+ 4232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:30.167811 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:30.167814 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:30.167843 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:30.171380 (+ 3537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:30.172129 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:30.172133 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:30.172135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:30.172315 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:30.172723 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:30.172727 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:30.172729 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:30.173000 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:30.173178 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:30.173572 (+ 394us) server_negotiation.cc:299] Negotiation successful
0809 12:16:30.173781 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":994,"thread_start_us":871,"threads_started":1}
I20260809 12:16:30.180190 21833 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:35762:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 12:16:30.181572 21833 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:16:30.189819 22022 tablet_service.cc:1467] Processing CreateTablet for tablet b5bc73aa9a4d4112928f77bd1dd079a3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=072e53535a744b83a64392880b7a03a7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:16:30.190490 22022 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5bc73aa9a4d4112928f77bd1dd079a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:30.190595 22208 tablet_service.cc:1467] Processing CreateTablet for tablet b5bc73aa9a4d4112928f77bd1dd079a3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=072e53535a744b83a64392880b7a03a7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:16:30.191267 22208 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5bc73aa9a4d4112928f77bd1dd079a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:30.191036 22407 tablet_service.cc:1467] Processing CreateTablet for tablet b5bc73aa9a4d4112928f77bd1dd079a3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=072e53535a744b83a64392880b7a03a7]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:16:30.191699 22407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b5bc73aa9a4d4112928f77bd1dd079a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:30.199416 22525 tablet_bootstrap.cc:492] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2: Bootstrap starting.
I20260809 12:16:30.199462 22527 tablet_bootstrap.cc:492] T b5bc73aa9a4d4112928f77bd1dd079a3 P ae8d3b2563e64040bd6b49e50f3b2c9a: Bootstrap starting.
I20260809 12:16:30.202329 22527 tablet_bootstrap.cc:654] T b5bc73aa9a4d4112928f77bd1dd079a3 P ae8d3b2563e64040bd6b49e50f3b2c9a: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:30.202308 22525 tablet_bootstrap.cc:654] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:30.202903 22523 tablet_bootstrap.cc:492] T b5bc73aa9a4d4112928f77bd1dd079a3 P f2f8a12891e24319b23d664cd5cf4917: Bootstrap starting.
I20260809 12:16:30.206161 22523 tablet_bootstrap.cc:654] T b5bc73aa9a4d4112928f77bd1dd079a3 P f2f8a12891e24319b23d664cd5cf4917: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:30.212285 22527 tablet_bootstrap.cc:492] T b5bc73aa9a4d4112928f77bd1dd079a3 P ae8d3b2563e64040bd6b49e50f3b2c9a: No bootstrap required, opened a new log
I20260809 12:16:30.212688 22527 ts_tablet_manager.cc:1397] T b5bc73aa9a4d4112928f77bd1dd079a3 P ae8d3b2563e64040bd6b49e50f3b2c9a: Time spent bootstrapping tablet: real 0.013s user 0.004s sys 0.005s
I20260809 12:16:30.214172 22527 raft_consensus.cc:348] T b5bc73aa9a4d4112928f77bd1dd079a3 P ae8d3b2563e64040bd6b49e50f3b2c9a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } }
I20260809 12:16:30.214588 22527 raft_consensus.cc:374] T b5bc73aa9a4d4112928f77bd1dd079a3 P ae8d3b2563e64040bd6b49e50f3b2c9a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:30.214672 22525 tablet_bootstrap.cc:492] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2: No bootstrap required, opened a new log
I20260809 12:16:30.214833 22527 raft_consensus.cc:729] T b5bc73aa9a4d4112928f77bd1dd079a3 P ae8d3b2563e64040bd6b49e50f3b2c9a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ae8d3b2563e64040bd6b49e50f3b2c9a, State: Initialized, Role: FOLLOWER
I20260809 12:16:30.215035 22525 ts_tablet_manager.cc:1397] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2: Time spent bootstrapping tablet: real 0.016s user 0.005s sys 0.008s
I20260809 12:16:30.215397 22527 consensus_queue.cc:260] T b5bc73aa9a4d4112928f77bd1dd079a3 P ae8d3b2563e64040bd6b49e50f3b2c9a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } }
I20260809 12:16:30.216095 22531 tablet_replica.cc:439] T b5bc73aa9a4d4112928f77bd1dd079a3 P ae8d3b2563e64040bd6b49e50f3b2c9a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } } }
I20260809 12:16:30.216305 22525 raft_consensus.cc:348] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } }
I20260809 12:16:30.216521 22531 tablet_replica.cc:442] T b5bc73aa9a4d4112928f77bd1dd079a3 P ae8d3b2563e64040bd6b49e50f3b2c9a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:16:30.216663 22525 raft_consensus.cc:374] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:30.216794 22525 raft_consensus.cc:729] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b107dcbd8c3e4b6a9951ff6b7494c4a2, State: Initialized, Role: FOLLOWER
I20260809 12:16:30.217325 22525 consensus_queue.cc:260] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } }
I20260809 12:16:30.218215 22532 tablet_replica.cc:439] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } } }
I20260809 12:16:30.218992 22532 tablet_replica.cc:442] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:16:30.219811 22525 ts_tablet_manager.cc:1428] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2: Time spent starting tablet: real 0.005s user 0.003s sys 0.000s
I20260809 12:16:30.221755 22523 tablet_bootstrap.cc:492] T b5bc73aa9a4d4112928f77bd1dd079a3 P f2f8a12891e24319b23d664cd5cf4917: No bootstrap required, opened a new log
I20260809 12:16:30.222146 22523 ts_tablet_manager.cc:1397] T b5bc73aa9a4d4112928f77bd1dd079a3 P f2f8a12891e24319b23d664cd5cf4917: Time spent bootstrapping tablet: real 0.019s user 0.005s sys 0.006s
I20260809 12:16:30.222612 22527 ts_tablet_manager.cc:1428] T b5bc73aa9a4d4112928f77bd1dd079a3 P ae8d3b2563e64040bd6b49e50f3b2c9a: Time spent starting tablet: real 0.010s user 0.004s sys 0.000s
I20260809 12:16:30.225651 22523 raft_consensus.cc:348] T b5bc73aa9a4d4112928f77bd1dd079a3 P f2f8a12891e24319b23d664cd5cf4917 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } }
I20260809 12:16:30.226058 22523 raft_consensus.cc:374] T b5bc73aa9a4d4112928f77bd1dd079a3 P f2f8a12891e24319b23d664cd5cf4917 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:30.226277 22523 raft_consensus.cc:729] T b5bc73aa9a4d4112928f77bd1dd079a3 P f2f8a12891e24319b23d664cd5cf4917 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2f8a12891e24319b23d664cd5cf4917, State: Initialized, Role: FOLLOWER
I20260809 12:16:30.226841 22523 consensus_queue.cc:260] T b5bc73aa9a4d4112928f77bd1dd079a3 P f2f8a12891e24319b23d664cd5cf4917 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } }
I20260809 12:16:30.228073 22523 ts_tablet_manager.cc:1428] T b5bc73aa9a4d4112928f77bd1dd079a3 P f2f8a12891e24319b23d664cd5cf4917: Time spent starting tablet: real 0.006s user 0.000s sys 0.003s
I20260809 12:16:30.227727 22548 tablet_replica.cc:439] T b5bc73aa9a4d4112928f77bd1dd079a3 P f2f8a12891e24319b23d664cd5cf4917: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } } }
I20260809 12:16:30.228333 22548 tablet_replica.cc:442] T b5bc73aa9a4d4112928f77bd1dd079a3 P f2f8a12891e24319b23d664cd5cf4917: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:16:30.268848 22532 raft_consensus.cc:482] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:16:30.269227 22532 raft_consensus.cc:504] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } }
I20260809 12:16:30.272524 22532 leader_election.cc:288] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ae8d3b2563e64040bd6b49e50f3b2c9a (127.19.183.194:41741), f2f8a12891e24319b23d664cd5cf4917 (127.19.183.195:33047)
I20260809 12:16:30.288388 22520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:30.274046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:52993 (local address 127.19.183.195:33047)
0809 12:16:30.274308 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:30.274313 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:30.275235 (+ 922us) server_negotiation.cc:407] Connection header received
0809 12:16:30.275596 (+ 361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:30.275603 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:30.275703 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:30.275796 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:30.277597 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.279099 (+ 1502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.284441 (+ 5342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.286509 (+ 2068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.286724 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:30.286807 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:30.287725 (+ 918us) server_negotiation.cc:299] Negotiation successful
0809 12:16:30.287932 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260809 12:16:30.289346 22578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:30.272606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:41741 (local address 127.19.183.193:60763)
0809 12:16:30.273807 (+ 1201us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:30.273829 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:30.275119 (+ 1290us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:30.275945 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:30.275953 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:30.276001 (+ 48us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:30.276902 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:30.276921 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.279488 (+ 2567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.279497 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:30.284620 (+ 5123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:30.284639 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.288017 (+ 3378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.288025 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:30.288302 (+ 277us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:30.288334 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 12:16:30.288431 (+ 97us) client_negotiation.cc:240] Negotiation successful
0809 12:16:30.288563 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1020,"mutex_wait_us":24,"thread_start_us":801,"threads_started":1}
I20260809 12:16:30.290020 22579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:30.273872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:33047 (local address 127.19.183.193:52993)
0809 12:16:30.275082 (+ 1210us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:30.275113 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:30.275380 (+ 267us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:30.276035 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:30.276039 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:30.276067 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:30.277257 (+ 1190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:30.277273 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.279304 (+ 2031us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.279309 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:30.284180 (+ 4871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:30.284197 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.287227 (+ 3030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.287246 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:30.287465 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:30.287525 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 12:16:30.288843 (+ 1318us) client_negotiation.cc:240] Negotiation successful
0809 12:16:30.289004 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1044,"mutex_wait_us":143,"thread_start_us":836,"threads_started":1}
I20260809 12:16:30.290493 22575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:30.272659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:60763 (local address 127.19.183.194:41741)
0809 12:16:30.273341 (+ 682us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:30.273348 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:30.274354 (+ 1006us) server_negotiation.cc:407] Connection header received
0809 12:16:30.275367 (+ 1013us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:30.275373 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:30.275490 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:30.275625 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:30.277215 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.279180 (+ 1965us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.284909 (+ 5729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.287252 (+ 2343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.287501 (+ 249us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:30.287626 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:30.289635 (+ 2009us) server_negotiation.cc:299] Negotiation successful
0809 12:16:30.289892 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":546,"thread_start_us":384,"threads_started":1}
I20260809 12:16:30.290994 22436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5bc73aa9a4d4112928f77bd1dd079a3" candidate_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2f8a12891e24319b23d664cd5cf4917" is_pre_election: true
I20260809 12:16:30.291366 22436 raft_consensus.cc:2454] T b5bc73aa9a4d4112928f77bd1dd079a3 P f2f8a12891e24319b23d664cd5cf4917 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b107dcbd8c3e4b6a9951ff6b7494c4a2 in term 0.
I20260809 12:16:30.291465 22244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5bc73aa9a4d4112928f77bd1dd079a3" candidate_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" is_pre_election: true
I20260809 12:16:30.291904 22244 raft_consensus.cc:2454] T b5bc73aa9a4d4112928f77bd1dd079a3 P ae8d3b2563e64040bd6b49e50f3b2c9a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b107dcbd8c3e4b6a9951ff6b7494c4a2 in term 0.
I20260809 12:16:30.292140 21955 leader_election.cc:302] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b107dcbd8c3e4b6a9951ff6b7494c4a2, f2f8a12891e24319b23d664cd5cf4917; no voters:
I20260809 12:16:30.292893 22532 raft_consensus.cc:2781] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:16:30.293213 22532 raft_consensus.cc:482] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:16:30.293411 22532 raft_consensus.cc:3037] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:30.299469 22532 raft_consensus.cc:504] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } }
I20260809 12:16:30.300823 22532 leader_election.cc:288] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [CANDIDATE]: Term 1 election: Requested vote from peers ae8d3b2563e64040bd6b49e50f3b2c9a (127.19.183.194:41741), f2f8a12891e24319b23d664cd5cf4917 (127.19.183.195:33047)
I20260809 12:16:30.301707 22436 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5bc73aa9a4d4112928f77bd1dd079a3" candidate_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2f8a12891e24319b23d664cd5cf4917"
I20260809 12:16:30.301715 22244 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b5bc73aa9a4d4112928f77bd1dd079a3" candidate_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a"
I20260809 12:16:30.302106 22436 raft_consensus.cc:3037] T b5bc73aa9a4d4112928f77bd1dd079a3 P f2f8a12891e24319b23d664cd5cf4917 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:30.302163 22244 raft_consensus.cc:3037] T b5bc73aa9a4d4112928f77bd1dd079a3 P ae8d3b2563e64040bd6b49e50f3b2c9a [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:30.305289 22436 raft_consensus.cc:2454] T b5bc73aa9a4d4112928f77bd1dd079a3 P f2f8a12891e24319b23d664cd5cf4917 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b107dcbd8c3e4b6a9951ff6b7494c4a2 in term 1.
I20260809 12:16:30.305934 21955 leader_election.cc:302] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b107dcbd8c3e4b6a9951ff6b7494c4a2, f2f8a12891e24319b23d664cd5cf4917; no voters:
I20260809 12:16:30.306447 22532 raft_consensus.cc:2781] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:16:30.306862 22244 raft_consensus.cc:2454] T b5bc73aa9a4d4112928f77bd1dd079a3 P ae8d3b2563e64040bd6b49e50f3b2c9a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b107dcbd8c3e4b6a9951ff6b7494c4a2 in term 1.
I20260809 12:16:30.307706 22532 raft_consensus.cc:686] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 1 LEADER]: Becoming Leader. State: Replica: b107dcbd8c3e4b6a9951ff6b7494c4a2, State: Running, Role: LEADER
I20260809 12:16:30.308336 22532 consensus_queue.cc:237] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } }
I20260809 12:16:30.311651 22580 tablet_replica.cc:439] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2: TxnStatusTablet state changed. Reason: New leader b107dcbd8c3e4b6a9951ff6b7494c4a2. Latest consensus state: current_term: 1 leader_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } } }
I20260809 12:16:30.312492 22580 tablet_replica.cc:442] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:16:30.313870 21833 catalog_manager.cc:5495] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2 reported cstate change: term changed from 0 to 1, leader changed from <none> to b107dcbd8c3e4b6a9951ff6b7494c4a2 (127.19.183.193). New cstate: current_term: 1 leader_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } health_report { overall_health: UNKNOWN } } }
I20260809 12:16:30.314210 22582 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 12:16:30.314697 22582 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 12:16:30.357879 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:30.346889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56196 (local address 127.19.183.193:36301)
0809 12:16:30.347134 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:30.347140 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:30.347164 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:16:30.347621 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:30.347627 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:30.347817 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:30.348056 (+ 239us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:16:30.348710 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.350551 (+ 1841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.353632 (+ 3081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.354998 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.355149 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:30.356005 (+ 856us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:16:30.356890 (+ 885us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:16:30.357057 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:30.357222 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 12:16:30.357362 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260809 12:16:30.357969 22568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:30.346599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:36301 (local address 127.0.0.1:56196)
0809 12:16:30.346933 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:30.346967 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:30.347155 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:30.348103 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:30.348109 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:30.348125 (+ 16us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:16:30.348495 (+ 370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:30.348510 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.350814 (+ 2304us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.350821 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:30.353385 (+ 2564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:30.353405 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.355225 (+ 1820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.355234 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:30.355646 (+ 412us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:30.355735 (+ 89us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:16:30.357028 (+ 1293us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:16:30.357067 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 12:16:30.357175 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 12:16:30.357439 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 12:16:30.365854 22244 raft_consensus.cc:1261] T b5bc73aa9a4d4112928f77bd1dd079a3 P ae8d3b2563e64040bd6b49e50f3b2c9a [term 1 FOLLOWER]: Refusing update from remote peer b107dcbd8c3e4b6a9951ff6b7494c4a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:16:30.366883 22532 consensus_queue.cc:1035] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:16:30.367607 22436 raft_consensus.cc:1261] T b5bc73aa9a4d4112928f77bd1dd079a3 P f2f8a12891e24319b23d664cd5cf4917 [term 1 FOLLOWER]: Refusing update from remote peer b107dcbd8c3e4b6a9951ff6b7494c4a2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:16:30.368743 22532 consensus_queue.cc:1035] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:16:30.380506 22531 tablet_replica.cc:439] T b5bc73aa9a4d4112928f77bd1dd079a3 P ae8d3b2563e64040bd6b49e50f3b2c9a: TxnStatusTablet state changed. Reason: New leader b107dcbd8c3e4b6a9951ff6b7494c4a2. Latest consensus state: current_term: 1 leader_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } } }
I20260809 12:16:30.381186 22531 tablet_replica.cc:442] T b5bc73aa9a4d4112928f77bd1dd079a3 P ae8d3b2563e64040bd6b49e50f3b2c9a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:16:30.387194 22532 tablet_replica.cc:439] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } } }
I20260809 12:16:30.387826 22532 tablet_replica.cc:442] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:16:30.388955 22540 mvcc.cc:205] Tried to move back new op lower bound from 7316593829324681216 to 7316593829108097024. Current Snapshot: MvccSnapshot[applied={T|T < 7316593829324681216 or (T in {7316593829324681216})}]
I20260809 12:16:30.389178 22531 tablet_replica.cc:439] T b5bc73aa9a4d4112928f77bd1dd079a3 P ae8d3b2563e64040bd6b49e50f3b2c9a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } } }
I20260809 12:16:30.389665 22531 tablet_replica.cc:442] T b5bc73aa9a4d4112928f77bd1dd079a3 P ae8d3b2563e64040bd6b49e50f3b2c9a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:16:30.415952 22548 tablet_replica.cc:439] T b5bc73aa9a4d4112928f77bd1dd079a3 P f2f8a12891e24319b23d664cd5cf4917: TxnStatusTablet state changed. Reason: New leader b107dcbd8c3e4b6a9951ff6b7494c4a2. Latest consensus state: current_term: 1 leader_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } } }
I20260809 12:16:30.416563 22548 tablet_replica.cc:442] T b5bc73aa9a4d4112928f77bd1dd079a3 P f2f8a12891e24319b23d664cd5cf4917: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:16:30.422333 22580 tablet_replica.cc:439] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } } }
I20260809 12:16:30.422901 22580 tablet_replica.cc:442] T b5bc73aa9a4d4112928f77bd1dd079a3 P b107dcbd8c3e4b6a9951ff6b7494c4a2: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:16:30.429524 22548 tablet_replica.cc:439] T b5bc73aa9a4d4112928f77bd1dd079a3 P f2f8a12891e24319b23d664cd5cf4917: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 } } peers { permanent_uuid: "ae8d3b2563e64040bd6b49e50f3b2c9a" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41741 } } peers { permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 } } }
I20260809 12:16:30.429965 22548 tablet_replica.cc:442] T b5bc73aa9a4d4112928f77bd1dd079a3 P f2f8a12891e24319b23d664cd5cf4917: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:16:30.437870 22575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:30.413299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59290 (local address 127.19.183.194:41741)
0809 12:16:30.413638 (+ 339us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:30.413644 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:30.413891 (+ 247us) server_negotiation.cc:407] Connection header received
0809 12:16:30.415640 (+ 1749us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:30.415648 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:30.415761 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:30.415946 (+ 185us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:16:30.417044 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.418407 (+ 1363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.432840 (+ 14433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.434244 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.434424 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:30.435751 (+ 1327us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:16:30.436749 (+ 998us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:16:30.436906 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:30.437230 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 12:16:30.437395 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 12:16:30.462971 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:30.454143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:45315 (local address 127.19.183.193:36301)
0809 12:16:30.454428 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:30.454434 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:30.455978 (+ 1544us) server_negotiation.cc:407] Connection header received
0809 12:16:30.456257 (+ 279us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:30.456261 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:30.456330 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:30.456433 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:16:30.457582 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.458309 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.460398 (+ 2089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.461176 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.461342 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:30.461899 (+ 557us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:16:30.462058 (+ 159us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:16:30.462163 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:30.462310 (+ 147us) server_negotiation.cc:299] Negotiation successful
0809 12:16:30.462390 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:16:30.463246 22604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:30.453961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:36301 (local address 127.19.183.194:45315)
0809 12:16:30.455816 (+ 1855us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:30.455833 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:30.456093 (+ 260us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:30.456488 (+ 395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:30.456493 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:30.456507 (+ 14us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:16:30.457321 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:30.457336 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.458450 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.458455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:30.460245 (+ 1790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:30.460259 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.461459 (+ 1200us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.461466 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:30.461654 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:30.461711 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:16:30.462169 (+ 458us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:16:30.462183 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 12:16:30.462285 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 12:16:30.462423 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1687,"thread_start_us":1421,"threads_started":1}
I20260809 12:16:30.478183 22436 raft_consensus.cc:1261] T c1155315e70047cd8bd796d04b977895 P f2f8a12891e24319b23d664cd5cf4917 [term 1 FOLLOWER]: Refusing update from remote peer ae8d3b2563e64040bd6b49e50f3b2c9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:16:30.478493 22047 raft_consensus.cc:1261] T c1155315e70047cd8bd796d04b977895 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 1 FOLLOWER]: Refusing update from remote peer ae8d3b2563e64040bd6b49e50f3b2c9a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:16:30.479269 22561 consensus_queue.cc:1035] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2f8a12891e24319b23d664cd5cf4917" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 33047 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 12:16:30.480262 22531 consensus_queue.cc:1035] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b107dcbd8c3e4b6a9951ff6b7494c4a2" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:16:30.514564 22590 mvcc.cc:205] Tried to move back new op lower bound from 7316593829788553216 to 7316593828331405312. Current Snapshot: MvccSnapshot[applied={T|T < 7316593829788553216}]
I20260809 12:16:30.565306 22575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:30.548781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:57603 (local address 127.19.183.194:41741)
0809 12:16:30.549001 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:30.549007 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:30.556231 (+ 7224us) server_negotiation.cc:407] Connection header received
0809 12:16:30.556617 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:30.556622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:30.556764 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:30.556836 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:16:30.558705 (+ 1869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.559791 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.562899 (+ 3108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.563516 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.563693 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:30.564319 (+ 626us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:16:30.564443 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:16:30.564574 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:30.564757 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 12:16:30.564854 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260809 12:16:30.566504 22609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:30.548916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:41741 (local address 127.19.183.193:57603)
0809 12:16:30.556122 (+ 7206us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:30.556145 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:30.556416 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:30.557069 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:30.557073 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:30.557086 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:16:30.558398 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:30.558416 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.559937 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.559941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:30.562675 (+ 2734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:30.562693 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.563822 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.563830 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:30.564036 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:30.564091 (+ 55us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:16:30.564576 (+ 485us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:16:30.564629 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 12:16:30.564729 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 12:16:30.564881 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6866,"thread_start_us":6446,"threads_started":1}
I20260809 12:16:30.640945 21832 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:16:30.641489 21832 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35758:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:16:30.646566 21832 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c70e29d6c6f742c781abe41e95fc5fcc: Sending DeleteTablet for 3 replicas of tablet c1155315e70047cd8bd796d04b977895
I20260809 12:16:30.647755 22022 tablet_service.cc:1514] Processing DeleteTablet for tablet c1155315e70047cd8bd796d04b977895 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:16:30 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:56188
I20260809 12:16:30.649330 22208 tablet_service.cc:1514] Processing DeleteTablet for tablet c1155315e70047cd8bd796d04b977895 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:16:30 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:59282
I20260809 12:16:30.657811 22407 tablet_service.cc:1514] Processing DeleteTablet for tablet c1155315e70047cd8bd796d04b977895 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:16:30 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:58876
I20260809 12:16:30.658962 22613 tablet_replica.cc:330] T c1155315e70047cd8bd796d04b977895 P b107dcbd8c3e4b6a9951ff6b7494c4a2: stopping tablet replica
I20260809 12:16:30.659413 22613 raft_consensus.cc:2229] T c1155315e70047cd8bd796d04b977895 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:16:30.660090 22613 raft_consensus.cc:2258] T c1155315e70047cd8bd796d04b977895 P b107dcbd8c3e4b6a9951ff6b7494c4a2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:16:30.661758 22613 ts_tablet_manager.cc:1905] T c1155315e70047cd8bd796d04b977895 P b107dcbd8c3e4b6a9951ff6b7494c4a2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:16:30.667869 22614 tablet_replica.cc:330] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a: stopping tablet replica
I20260809 12:16:30.668170 22614 raft_consensus.cc:2229] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:16:30.668958 22615 tablet_replica.cc:330] T c1155315e70047cd8bd796d04b977895 P f2f8a12891e24319b23d664cd5cf4917: stopping tablet replica
I20260809 12:16:30.669052 22614 raft_consensus.cc:2258] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:16:30.669286 22615 raft_consensus.cc:2229] T c1155315e70047cd8bd796d04b977895 P f2f8a12891e24319b23d664cd5cf4917 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:16:30.669989 22615 raft_consensus.cc:2258] T c1155315e70047cd8bd796d04b977895 P f2f8a12891e24319b23d664cd5cf4917 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:16:30.671767 22615 ts_tablet_manager.cc:1905] T c1155315e70047cd8bd796d04b977895 P f2f8a12891e24319b23d664cd5cf4917: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:16:30.675287 22614 ts_tablet_manager.cc:1905] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:16:30.683411 22613 ts_tablet_manager.cc:1918] T c1155315e70047cd8bd796d04b977895 P b107dcbd8c3e4b6a9951ff6b7494c4a2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:16:30.683749 22613 log.cc:1196] T c1155315e70047cd8bd796d04b977895 P b107dcbd8c3e4b6a9951ff6b7494c4a2: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/c1155315e70047cd8bd796d04b977895
I20260809 12:16:30.686421 22614 ts_tablet_manager.cc:1918] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:16:30.686863 22614 log.cc:1196] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/c1155315e70047cd8bd796d04b977895
I20260809 12:16:30.687911 22614 ts_tablet_manager.cc:1939] T c1155315e70047cd8bd796d04b977895 P ae8d3b2563e64040bd6b49e50f3b2c9a: Deleting consensus metadata
I20260809 12:16:30.691502 22613 ts_tablet_manager.cc:1939] T c1155315e70047cd8bd796d04b977895 P b107dcbd8c3e4b6a9951ff6b7494c4a2: Deleting consensus metadata
I20260809 12:16:30.693142 21811 catalog_manager.cc:4841] TS ae8d3b2563e64040bd6b49e50f3b2c9a (127.19.183.194:41741): tablet c1155315e70047cd8bd796d04b977895 (table test-table [id=677d3870f286425ca517eda84a0cdfc0]) successfully deleted
I20260809 12:16:30.693195 21813 catalog_manager.cc:4841] TS b107dcbd8c3e4b6a9951ff6b7494c4a2 (127.19.183.193:36301): tablet c1155315e70047cd8bd796d04b977895 (table test-table [id=677d3870f286425ca517eda84a0cdfc0]) successfully deleted
Aug 09 12:16:30 dist-test-slave-39l8 krb5kdc[21768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277789, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.183.193@KRBTEST.COM
I20260809 12:16:30.702956 22615 ts_tablet_manager.cc:1918] T c1155315e70047cd8bd796d04b977895 P f2f8a12891e24319b23d664cd5cf4917: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:16:30.703285 22615 log.cc:1196] T c1155315e70047cd8bd796d04b977895 P f2f8a12891e24319b23d664cd5cf4917: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/c1155315e70047cd8bd796d04b977895
I20260809 12:16:30.704193 22615 ts_tablet_manager.cc:1939] T c1155315e70047cd8bd796d04b977895 P f2f8a12891e24319b23d664cd5cf4917: Deleting consensus metadata
I20260809 12:16:30.711863 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:30.668919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56204 (local address 127.19.183.193:36301)
0809 12:16:30.669284 (+ 365us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:30.669291 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:30.677644 (+ 8353us) server_negotiation.cc:407] Connection header received
0809 12:16:30.678315 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:30.678321 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:30.678405 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:30.678525 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:30.683062 (+ 4537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.684973 (+ 1911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.687949 (+ 2976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.688922 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.689065 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:30.698321 (+ 9256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:30.698350 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:30.698387 (+ 37us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:30.698467 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:30.705430 (+ 6963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:30.706786 (+ 1356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:30.706797 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:30.706810 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:30.707058 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:30.707746 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:30.707752 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:30.707755 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:30.708604 (+ 849us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:30.709339 (+ 735us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:30.710951 (+ 1612us) server_negotiation.cc:299] Negotiation successful
0809 12:16:30.711253 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177}
W20260809 12:16:30.713413 22107 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56204
I20260809 12:16:30.714375 21811 catalog_manager.cc:4841] TS f2f8a12891e24319b23d664cd5cf4917 (127.19.183.195:33047): tablet c1155315e70047cd8bd796d04b977895 (table test-table [id=677d3870f286425ca517eda84a0cdfc0]) successfully deleted
I20260809 12:16:30.764190 22622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:30.724477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35774 (local address 127.19.183.254:33911)
0809 12:16:30.733752 (+ 9275us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:30.733762 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:30.737993 (+ 4231us) server_negotiation.cc:407] Connection header received
0809 12:16:30.738289 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:30.738295 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:30.738451 (+ 156us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:30.738598 (+ 147us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:30.742092 (+ 3494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.744179 (+ 2087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.746755 (+ 2576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.747932 (+ 1177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.748041 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:30.753703 (+ 5662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:30.753710 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:30.753716 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:30.753794 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:30.759686 (+ 5892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:30.760841 (+ 1155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:30.760847 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:30.760850 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:30.761095 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:30.761724 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:30.761729 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:30.761733 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:30.762230 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:30.762569 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:30.763322 (+ 753us) server_negotiation.cc:299] Negotiation successful
0809 12:16:30.763605 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9116,"thread_start_us":8991,"threads_started":1}
W20260809 12:16:30.766604 21832 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35774
I20260809 12:16:30.769821 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 21933
I20260809 12:16:30.804454 22518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:30.771588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56218 (local address 127.19.183.193:36301)
0809 12:16:30.771832 (+ 244us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:30.771838 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:30.782486 (+ 10648us) server_negotiation.cc:407] Connection header received
0809 12:16:30.783033 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:30.783039 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:30.783125 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:30.783235 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:30.785212 (+ 1977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.786604 (+ 1392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.788728 (+ 2124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.789586 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.789694 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:30.794123 (+ 4429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:30.794165 (+ 42us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:30.794170 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:30.794221 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:30.800010 (+ 5789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:30.801024 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:30.801030 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:30.801034 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:30.801283 (+ 249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:30.801918 (+ 635us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:30.801923 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:30.801927 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:30.802417 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:30.802795 (+ 378us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:30.803621 (+ 826us) server_negotiation.cc:299] Negotiation successful
0809 12:16:30.803925 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
W20260809 12:16:30.806491 22107 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56218
W20260809 12:16:30.807426 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 21933: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:16:30.807770 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 21933
I20260809 12:16:30.844847 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 22122
Aug 09 12:16:30 dist-test-slave-39l8 krb5kdc[21768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277789, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.183.194@KRBTEST.COM
I20260809 12:16:30.861413 22575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:30.845669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59292 (local address 127.19.183.194:41741)
0809 12:16:30.845860 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:30.845867 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:30.845982 (+ 115us) server_negotiation.cc:407] Connection header received
0809 12:16:30.846215 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:30.846221 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:30.846303 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:30.846416 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:30.847916 (+ 1500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.848787 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.850315 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.850824 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.850966 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:30.855273 (+ 4307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:30.855285 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:30.855300 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:30.855333 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:30.858467 (+ 3134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:30.859175 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:30.859185 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:30.859199 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:30.859346 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:30.859761 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:30.859765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:30.859767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:30.860170 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:30.860483 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:30.860789 (+ 306us) server_negotiation.cc:299] Negotiation successful
0809 12:16:30.860996 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 12:16:30.862172 22302 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59292
W20260809 12:16:30.862881 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 22122: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:16:30.863063 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 22122
I20260809 12:16:30.903342 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 22314
Aug 09 12:16:30 dist-test-slave-39l8 krb5kdc[21768](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277789, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.183.195@KRBTEST.COM
I20260809 12:16:30.919483 22626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:30.904213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58892 (local address 127.19.183.195:33047)
0809 12:16:30.905040 (+ 827us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:30.905045 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:30.905061 (+ 16us) server_negotiation.cc:407] Connection header received
0809 12:16:30.905115 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:30.905125 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:30.905195 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:30.905322 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:30.906606 (+ 1284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.907507 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.908930 (+ 1423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:30.909363 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:30.909501 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:30.913767 (+ 4266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:30.913783 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:30.913799 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:30.913837 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:30.916598 (+ 2761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:30.917327 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:30.917335 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:30.917338 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:30.917463 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:30.917842 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:30.917848 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:30.917853 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:30.918208 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:30.918522 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:30.918843 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 12:16:30.919067 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":736,"thread_start_us":597,"threads_started":1}
W20260809 12:16:30.920192 22494 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58892
W20260809 12:16:30.920853 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 22314: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:16:30.921016 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 22314
I20260809 12:16:30.953670 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 21788
I20260809 12:16:30.954835 21885 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:16:31.070331 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 21788
2026-08-09T12:16:31Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5136 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:16:31 dist-test-slave-39l8 krb5kdc[22631](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:16:31 dist-test-slave-39l8 krb5kdc[22631](info): set up 2 sockets
Aug 09 12:16:31 dist-test-slave-39l8 krb5kdc[22631](info): commencing operation
krb5kdc: starting...
W20260809 12:16:32.360411 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.237s user 0.005s sys 0.017s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:16:32 dist-test-slave-39l8 krb5kdc[22631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277792, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:16:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:16:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:16:32.572382 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:45709
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:36167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:45709
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:32.903707 22646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:32.903976 22646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:32.904083 22646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:32.910988 22646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:16:32.911087 22646 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 12:16:32.911134 22646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:32.911196 22646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:32.911247 22646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:16:32.911301 22646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:16:32.920953 22646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:36167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:45709
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:45709
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:32.922104 22646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:32.923373 22646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:32.934774 22653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:32.934813 22656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:32.934860 22654 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:32.936074 22646 server_base.cc:1034] running on GCE node
I20260809 12:16:32.936832 22646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:32.938632 22646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:32.939896 22646 hybrid_clock.cc:648] HybridClock initialized: now 1786277792939842 us; error 31 us; skew 500 ppm
I20260809 12:16:32.940245 22646 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:32 dist-test-slave-39l8 krb5kdc[22631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277792, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:32.943451 22646 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:16:32.944926 22646 webserver.cc:460] Webserver started at http://127.19.183.254:33907/ using document root <none> and password file <none>
I20260809 12:16:32.945624 22646 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:32.945715 22646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:32.945982 22646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:32.949033 22646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "d975166788fd4936a3544e7d854a0318"
format_stamp: "Formatted at 2026-08-09 12:16:32 on dist-test-slave-39l8"
server_key: "cfc62f3f27ec75c51e9e7ef4fb43e4d6"
server_key_iv: "51ecdf7a921c53c1f0ae5b57ca1cf31d"
server_key_version: "encryptionkey@0"
I20260809 12:16:32.949761 22646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "d975166788fd4936a3544e7d854a0318"
format_stamp: "Formatted at 2026-08-09 12:16:32 on dist-test-slave-39l8"
server_key: "cfc62f3f27ec75c51e9e7ef4fb43e4d6"
server_key_iv: "51ecdf7a921c53c1f0ae5b57ca1cf31d"
server_key_version: "encryptionkey@0"
I20260809 12:16:32.954905 22646 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.004s
I20260809 12:16:32.958192 22663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:32.959549 22646 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:16:32.959692 22646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "d975166788fd4936a3544e7d854a0318"
format_stamp: "Formatted at 2026-08-09 12:16:32 on dist-test-slave-39l8"
server_key: "cfc62f3f27ec75c51e9e7ef4fb43e4d6"
server_key_iv: "51ecdf7a921c53c1f0ae5b57ca1cf31d"
server_key_version: "encryptionkey@0"
I20260809 12:16:32.959865 22646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:32.981878 22646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:32.985787 22646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:32.986014 22646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:33.008256 22646 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:45709
I20260809 12:16:33.008291 22748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:45709 every 8 connection(s)
I20260809 12:16:33.009644 22646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:16:33.010435 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 22646
I20260809 12:16:33.010663 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:16:33.011507 20191 external_mini_cluster.cc:1428] Setting key e5ec05150dc65fef34b454ded169cefc
I20260809 12:16:33.014461 22749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 12:16:33 dist-test-slave-39l8 krb5kdc[22631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277792, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:33.027549 22749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318: Bootstrap starting.
I20260809 12:16:33.030951 22749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:33.032171 22749 log.cc:824] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318: Log is configured to *not* fsync() on all Append() calls
I20260809 12:16:33.035485 22749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318: No bootstrap required, opened a new log
I20260809 12:16:33.036044 22753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:33.013640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56648 (local address 127.19.183.254:45709)
0809 12:16:33.014574 (+ 934us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:33.014589 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:33.014639 (+ 50us) server_negotiation.cc:407] Connection header received
0809 12:16:33.017027 (+ 2388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:33.017055 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:33.017564 (+ 509us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:33.018004 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:33.019422 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:33.021574 (+ 2152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:33.022890 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:33.023902 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:33.024093 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:33.028382 (+ 4289us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:33.028390 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:33.028403 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:33.028443 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:33.031688 (+ 3245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:33.032528 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:33.032534 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:33.032540 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:33.032756 (+ 216us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:33.033200 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:33.033219 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:33.033225 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:33.033882 (+ 657us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:33.034111 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:33.034556 (+ 445us) server_negotiation.cc:299] Negotiation successful
0809 12:16:33.035005 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":671,"thread_start_us":438,"threads_started":1}
I20260809 12:16:33.043854 22749 raft_consensus.cc:348] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d975166788fd4936a3544e7d854a0318" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45709 } }
I20260809 12:16:33.044214 22749 raft_consensus.cc:374] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:33.044314 22749 raft_consensus.cc:729] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d975166788fd4936a3544e7d854a0318, State: Initialized, Role: FOLLOWER
I20260809 12:16:33.044904 22749 consensus_queue.cc:260] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d975166788fd4936a3544e7d854a0318" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45709 } }
I20260809 12:16:33.045137 22749 raft_consensus.cc:388] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:16:33.045303 22749 raft_consensus.cc:482] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:16:33.045472 22749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:33.047890 22749 raft_consensus.cc:504] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d975166788fd4936a3544e7d854a0318" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45709 } }
I20260809 12:16:33.048388 22749 leader_election.cc:302] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d975166788fd4936a3544e7d854a0318; no voters:
I20260809 12:16:33.048944 22749 leader_election.cc:288] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:16:33.049085 22755 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:16:33.049890 22755 raft_consensus.cc:686] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318 [term 1 LEADER]: Becoming Leader. State: Replica: d975166788fd4936a3544e7d854a0318, State: Running, Role: LEADER
I20260809 12:16:33.050510 22755 consensus_queue.cc:237] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d975166788fd4936a3544e7d854a0318" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45709 } }
I20260809 12:16:33.051151 22749 sys_catalog.cc:564] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:16:33.053680 22758 sys_catalog.cc:455] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d975166788fd4936a3544e7d854a0318. Latest consensus state: current_term: 1 leader_uuid: "d975166788fd4936a3544e7d854a0318" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d975166788fd4936a3544e7d854a0318" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45709 } } }
I20260809 12:16:33.054204 22758 sys_catalog.cc:458] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318 [sys.catalog]: This master's current role is: LEADER
I20260809 12:16:33.055040 22756 sys_catalog.cc:455] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d975166788fd4936a3544e7d854a0318" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d975166788fd4936a3544e7d854a0318" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45709 } } }
I20260809 12:16:33.055395 22756 sys_catalog.cc:458] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318 [sys.catalog]: This master's current role is: LEADER
W20260809 12:16:33.065932 22775 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 12:16:33.066170 22775 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 12:16:33.066761 22763 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:16:33.068981 22763 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:16:33.077459 22763 catalog_manager.cc:1347] Generated new cluster ID: 9053c678a6ee496ab2bb7e80a82160cc
I20260809 12:16:33.077564 22763 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:16:33.098788 22763 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:16:33.100059 22763 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:16:33.107393 22763 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318: Generated new TSK 0
I20260809 12:16:33.108176 22763 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
I20260809 12:16:33.172513 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:45709
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:36167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:33.538516 22782 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:33.538905 22782 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:33.539117 22782 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:33.550407 22782 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:33.550586 22782 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:33.550758 22782 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 12:16:33.550868 22782 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:16:33.550992 22782 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:16:33.565449 22782 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:36167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.183.254:45709
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:33.566815 22782 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:33.568162 22782 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:33.581063 22794 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:33.581172 22788 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:33.581095 22790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:33.584750 22782 server_base.cc:1034] running on GCE node
I20260809 12:16:33.585363 22782 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:33.586400 22782 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:33.587667 22782 hybrid_clock.cc:648] HybridClock initialized: now 1786277793587662 us; error 120 us; skew 500 ppm
I20260809 12:16:33.587981 22782 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:33 dist-test-slave-39l8 krb5kdc[22631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277793, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:33.591537 22782 init.cc:377] Logged in from keytab as kudu/127.19.183.193@KRBTEST.COM (short username kudu)
I20260809 12:16:33.593039 22782 webserver.cc:460] Webserver started at http://127.19.183.193:37569/ using document root <none> and password file <none>
I20260809 12:16:33.593717 22782 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:33.593822 22782 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:33.594125 22782 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:33.597013 22782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5"
format_stamp: "Formatted at 2026-08-09 12:16:33 on dist-test-slave-39l8"
server_key: "d072165a3718bda2ff1ae27ee4376643"
server_key_iv: "5e1db1ee94e049a3fe6d03fca5d1d9d3"
server_key_version: "encryptionkey@0"
I20260809 12:16:33.597723 22782 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5"
format_stamp: "Formatted at 2026-08-09 12:16:33 on dist-test-slave-39l8"
server_key: "d072165a3718bda2ff1ae27ee4376643"
server_key_iv: "5e1db1ee94e049a3fe6d03fca5d1d9d3"
server_key_version: "encryptionkey@0"
I20260809 12:16:33.602658 22782 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260809 12:16:33.605868 22810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:33.607285 22782 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:16:33.607425 22782 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5"
format_stamp: "Formatted at 2026-08-09 12:16:33 on dist-test-slave-39l8"
server_key: "d072165a3718bda2ff1ae27ee4376643"
server_key_iv: "5e1db1ee94e049a3fe6d03fca5d1d9d3"
server_key_version: "encryptionkey@0"
I20260809 12:16:33.607604 22782 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:33.624626 22782 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:33.628384 22782 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:33.628674 22782 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:33.631356 22782 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:33.631490 22782 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:33.631614 22782 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:33.631699 22782 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:16:33 dist-test-slave-39l8 krb5kdc[22631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277793, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:33.654913 22822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:33.631774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:49877 (local address 127.19.183.254:45709)
0809 12:16:33.632661 (+ 887us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:33.632665 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:33.633341 (+ 676us) server_negotiation.cc:407] Connection header received
0809 12:16:33.635493 (+ 2152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:33.635498 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:33.635588 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:33.635705 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:33.637743 (+ 2038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:33.638829 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:33.641429 (+ 2600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:33.642076 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:33.642220 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:33.647963 (+ 5743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:33.647968 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:33.647972 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:33.648005 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:33.651263 (+ 3258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:33.652233 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:33.652240 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:33.652243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:33.652408 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:33.652870 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:33.652876 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:33.652879 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:33.653185 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:33.653336 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:33.654057 (+ 721us) server_negotiation.cc:299] Negotiation successful
0809 12:16:33.654356 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":785,"thread_start_us":684,"threads_started":1}
I20260809 12:16:33.656454 22823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:33.632373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45709 (local address 127.19.183.193:49877)
0809 12:16:33.633191 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:33.633228 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:33.635217 (+ 1989us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:33.635930 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:33.635938 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:33.636689 (+ 751us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:33.637530 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:33.637566 (+ 36us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:33.639022 (+ 1456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:33.639032 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:33.641228 (+ 2196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:33.641245 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:33.643182 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:33.643202 (+ 20us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:33.643629 (+ 427us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:33.644379 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:33.644422 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:33.647711 (+ 3289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:33.651389 (+ 3678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:33.651401 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:33.651417 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:33.652084 (+ 667us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:33.652535 (+ 451us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:33.652542 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:33.652548 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:33.652754 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:33.653314 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:33.653322 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:33.653694 (+ 372us) client_negotiation.cc:769] Sending connection context
0809 12:16:33.653961 (+ 267us) client_negotiation.cc:240] Negotiation successful
0809 12:16:33.654790 (+ 829us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":586,"thread_start_us":345,"threads_started":1}
I20260809 12:16:33.678696 22822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:33.662967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:37423 (local address 127.19.183.254:45709)
0809 12:16:33.663188 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:33.663195 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:33.663850 (+ 655us) server_negotiation.cc:407] Connection header received
0809 12:16:33.664107 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:33.664110 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:33.664205 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:33.664284 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:33.665699 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:33.666401 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:33.668948 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:33.669484 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:33.669626 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:33.672468 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:33.672473 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:33.672477 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:33.672514 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:33.675699 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:33.676697 (+ 998us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:33.676702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:33.676706 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:33.676840 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:33.677239 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:33.677243 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:33.677245 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:33.677512 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:33.677692 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:33.678086 (+ 394us) server_negotiation.cc:299] Negotiation successful
0809 12:16:33.678303 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 12:16:33.678884 22887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:33.662898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45709 (local address 127.19.183.193:37423)
0809 12:16:33.663734 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:33.663758 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:33.663922 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:33.664351 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:33.664358 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:33.665061 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:33.665529 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:33.665545 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:33.666552 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:33.666560 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:33.668772 (+ 2212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:33.668793 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:33.669869 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:33.669878 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:33.670163 (+ 285us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:33.670924 (+ 761us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:33.670960 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:33.672255 (+ 1295us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:33.675827 (+ 3572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:33.675832 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:33.675836 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:33.676562 (+ 726us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:33.676959 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:33.676964 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:33.676968 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:33.677109 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:33.677648 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:33.677653 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:33.677832 (+ 179us) client_negotiation.cc:769] Sending connection context
0809 12:16:33.677967 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 12:16:33.678303 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":642,"thread_start_us":401,"threads_started":1}
I20260809 12:16:33.692430 22782 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:37075
I20260809 12:16:33.692471 22972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:37075 every 8 connection(s)
I20260809 12:16:33.693707 22782 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:16:33.695688 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 22782
I20260809 12:16:33.695931 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:16:33.696077 22973 heartbeater.cc:347] Connected to a master server at 127.19.183.254:45709
I20260809 12:16:33.696326 22973 heartbeater.cc:464] Registering TS with master...
I20260809 12:16:33.696705 20191 external_mini_cluster.cc:1428] Setting key fa583c701d329788d530c854ce1d4c69
I20260809 12:16:33.697018 22973 heartbeater.cc:511] Master 127.19.183.254:45709 requested a full tablet report, sending...
I20260809 12:16:33.699262 22682 ts_manager.cc:194] Registered new tserver with Master: dfc9eb51ad9848f08a94c57a60c7c1c5 (127.19.183.193:37075)
I20260809 12:16:33.700706 22682 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:49877
WARNING: no policy specified for kudu/127.19.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
I20260809 12:16:33.745471 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:45709
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:36167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:34.081604 22977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:34.081935 22977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:34.082127 22977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:34.089171 22977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:34.089339 22977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:34.089484 22977 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 12:16:34.089574 22977 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:16:34.089725 22977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:16:34.099452 22977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:36167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.183.254:45709
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:34.100703 22977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:34.101962 22977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:34.114987 22982 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:34.114992 22986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:34.115283 22983 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:34.115962 22977 server_base.cc:1034] running on GCE node
I20260809 12:16:34.116611 22977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:34.117543 22977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:34.118803 22977 hybrid_clock.cc:648] HybridClock initialized: now 1786277794118740 us; error 60 us; skew 500 ppm
I20260809 12:16:34.119127 22977 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:34 dist-test-slave-39l8 krb5kdc[22631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277794, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:34.128194 22977 init.cc:377] Logged in from keytab as kudu/127.19.183.194@KRBTEST.COM (short username kudu)
I20260809 12:16:34.129732 22977 webserver.cc:460] Webserver started at http://127.19.183.194:43583/ using document root <none> and password file <none>
I20260809 12:16:34.130437 22977 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:34.130589 22977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:34.130962 22977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:34.134023 22977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "6183387a03104ef6abd3836001c0dcdd"
format_stamp: "Formatted at 2026-08-09 12:16:34 on dist-test-slave-39l8"
server_key: "6d65a573826f17f98fee0aff850acaeb"
server_key_iv: "afed074c8aa4a4da331ec63039e61cd3"
server_key_version: "encryptionkey@0"
I20260809 12:16:34.134867 22977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "6183387a03104ef6abd3836001c0dcdd"
format_stamp: "Formatted at 2026-08-09 12:16:34 on dist-test-slave-39l8"
server_key: "6d65a573826f17f98fee0aff850acaeb"
server_key_iv: "afed074c8aa4a4da331ec63039e61cd3"
server_key_version: "encryptionkey@0"
I20260809 12:16:34.139940 22977 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 12:16:34.143245 22998 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:34.144409 22977 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:16:34.144584 22977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "6183387a03104ef6abd3836001c0dcdd"
format_stamp: "Formatted at 2026-08-09 12:16:34 on dist-test-slave-39l8"
server_key: "6d65a573826f17f98fee0aff850acaeb"
server_key_iv: "afed074c8aa4a4da331ec63039e61cd3"
server_key_version: "encryptionkey@0"
I20260809 12:16:34.144786 22977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:34.168241 22977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:34.172077 22977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:34.172384 22977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:34.175040 22977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:34.175205 22977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:34.175346 22977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:34.175442 22977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:16:34 dist-test-slave-39l8 krb5kdc[22631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277794, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:34.202037 22822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:34.175968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:58933 (local address 127.19.183.254:45709)
0809 12:16:34.176180 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:34.176187 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:34.177766 (+ 1579us) server_negotiation.cc:407] Connection header received
0809 12:16:34.181125 (+ 3359us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:34.181131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:34.181234 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:34.181354 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:34.184228 (+ 2874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.185497 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.188316 (+ 2819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.188850 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.188997 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:34.194673 (+ 5676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:34.194678 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:34.194681 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:34.194792 (+ 111us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:34.198405 (+ 3613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:34.199547 (+ 1142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:34.199554 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:34.199558 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:34.199712 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:34.200209 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:34.200215 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:34.200218 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:34.200511 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:34.200688 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:34.201432 (+ 744us) server_negotiation.cc:299] Negotiation successful
0809 12:16:34.201666 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 12:16:34.203352 23010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:34.176596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45709 (local address 127.19.183.194:58933)
0809 12:16:34.177538 (+ 942us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:34.177602 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:34.180694 (+ 3092us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:34.181681 (+ 987us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:34.181694 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:34.182876 (+ 1182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:34.183954 (+ 1078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:34.184001 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.185725 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.185734 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:34.188140 (+ 2406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:34.188154 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.189188 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.189203 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:34.189553 (+ 350us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:34.190396 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:34.190440 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:34.194410 (+ 3970us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:34.198631 (+ 4221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:34.198644 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:34.198668 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:34.199330 (+ 662us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:34.199838 (+ 508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:34.199843 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:34.199850 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:34.200072 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:34.200660 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:34.200669 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:34.201041 (+ 372us) client_negotiation.cc:769] Sending connection context
0809 12:16:34.201311 (+ 270us) client_negotiation.cc:240] Negotiation successful
0809 12:16:34.201868 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":630,"thread_start_us":368,"threads_started":1}
I20260809 12:16:34.222108 22977 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:42423
I20260809 12:16:34.222137 23159 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:42423 every 8 connection(s)
I20260809 12:16:34.223646 22977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:16:34.225952 23160 heartbeater.cc:347] Connected to a master server at 127.19.183.254:45709
I20260809 12:16:34.226222 23160 heartbeater.cc:464] Registering TS with master...
I20260809 12:16:34.226569 22822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:34.210591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:47845 (local address 127.19.183.254:45709)
0809 12:16:34.210875 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:34.210883 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:34.211815 (+ 932us) server_negotiation.cc:407] Connection header received
0809 12:16:34.212078 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:34.212105 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:34.212195 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:34.212291 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:34.213656 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.214391 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.216632 (+ 2241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.217197 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.217338 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:34.220263 (+ 2925us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:34.220268 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:34.220271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:34.220304 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:34.223280 (+ 2976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:34.224149 (+ 869us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:34.224157 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:34.224163 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:34.224399 (+ 236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:34.224909 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:34.224913 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:34.224916 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:34.225352 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:34.225504 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:34.226000 (+ 496us) server_negotiation.cc:299] Negotiation successful
0809 12:16:34.226215 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 12:16:34.226734 23119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:34.210619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45709 (local address 127.19.183.194:47845)
0809 12:16:34.211677 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:34.211699 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:34.211883 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:34.212350 (+ 467us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:34.212355 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:34.213023 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:34.213464 (+ 441us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:34.213488 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.214549 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.214568 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:34.216318 (+ 1750us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:34.216331 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.217436 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.217446 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:34.217684 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:34.218461 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:34.218490 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:34.220054 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:34.223416 (+ 3362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:34.223421 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:34.223425 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:34.223986 (+ 561us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:34.224552 (+ 566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:34.224557 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:34.224560 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:34.224725 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:34.225479 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:34.225508 (+ 29us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:34.225709 (+ 201us) client_negotiation.cc:769] Sending connection context
0809 12:16:34.225884 (+ 175us) client_negotiation.cc:240] Negotiation successful
0809 12:16:34.226245 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"thread_start_us":647,"threads_started":1}
I20260809 12:16:34.227079 23160 heartbeater.cc:511] Master 127.19.183.254:45709 requested a full tablet report, sending...
I20260809 12:16:34.227125 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 22977
I20260809 12:16:34.227298 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:16:34.227886 20191 external_mini_cluster.cc:1428] Setting key 474f8f59a8453dd3a5c420d5af20e0c1
I20260809 12:16:34.228659 22682 ts_manager.cc:194] Registered new tserver with Master: 6183387a03104ef6abd3836001c0dcdd (127.19.183.194:42423)
I20260809 12:16:34.229944 22682 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:58933
WARNING: no policy specified for kudu/127.19.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
I20260809 12:16:34.279585 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:45709
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:36167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:34.618893 23164 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:34.619196 23164 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:34.619371 23164 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:34.626080 23164 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:34.626224 23164 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:34.626399 23164 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 12:16:34.626494 23164 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:16:34.626605 23164 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:16:34.635790 23164 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:36167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.183.254:45709
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:34.637079 23164 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:34.638386 23164 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:34.652199 23171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:34.652343 23169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:34.652350 23174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:34.653191 23164 server_base.cc:1034] running on GCE node
I20260809 12:16:34.653702 23164 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:34.654615 23164 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:34.655817 23164 hybrid_clock.cc:648] HybridClock initialized: now 1786277794655771 us; error 44 us; skew 500 ppm
I20260809 12:16:34.656095 23164 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:34 dist-test-slave-39l8 krb5kdc[22631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277794, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:34.659576 23164 init.cc:377] Logged in from keytab as kudu/127.19.183.195@KRBTEST.COM (short username kudu)
I20260809 12:16:34.661175 23164 webserver.cc:460] Webserver started at http://127.19.183.195:34767/ using document root <none> and password file <none>
I20260809 12:16:34.661805 23164 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:34.661907 23164 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:34.662201 23164 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:34.665635 23164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "4f3869b285954006b6ddf897a37b918f"
format_stamp: "Formatted at 2026-08-09 12:16:34 on dist-test-slave-39l8"
server_key: "88dbfd38c68015a2e57af064d7f9cd0e"
server_key_iv: "151cd0194ced94a0443008023eb1369b"
server_key_version: "encryptionkey@0"
I20260809 12:16:34.666443 23164 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "4f3869b285954006b6ddf897a37b918f"
format_stamp: "Formatted at 2026-08-09 12:16:34 on dist-test-slave-39l8"
server_key: "88dbfd38c68015a2e57af064d7f9cd0e"
server_key_iv: "151cd0194ced94a0443008023eb1369b"
server_key_version: "encryptionkey@0"
I20260809 12:16:34.671919 23164 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260809 12:16:34.675693 23186 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:34.677065 23164 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 12:16:34.677258 23164 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "4f3869b285954006b6ddf897a37b918f"
format_stamp: "Formatted at 2026-08-09 12:16:34 on dist-test-slave-39l8"
server_key: "88dbfd38c68015a2e57af064d7f9cd0e"
server_key_iv: "151cd0194ced94a0443008023eb1369b"
server_key_version: "encryptionkey@0"
I20260809 12:16:34.677479 23164 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:34.690160 23164 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:34.695379 23164 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:34.695691 23164 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:34.699344 23164 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:34.699476 23164 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.001s user 0.000s sys 0.000s
I20260809 12:16:34.699608 23164 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:34.699698 23164 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:34.703459 22973 heartbeater.cc:503] Master 127.19.183.254:45709 was elected leader, sending a full tablet report...
Aug 09 12:16:34 dist-test-slave-39l8 krb5kdc[22631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277794, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:34.724748 22822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:34.699060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:42035 (local address 127.19.183.254:45709)
0809 12:16:34.699284 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:34.699291 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:34.700799 (+ 1508us) server_negotiation.cc:407] Connection header received
0809 12:16:34.702890 (+ 2091us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:34.702897 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:34.702985 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:34.703105 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:34.706188 (+ 3083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.707297 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.709875 (+ 2578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.710541 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.710694 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:34.715858 (+ 5164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:34.715867 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:34.715875 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:34.715927 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:34.720094 (+ 4167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:34.721194 (+ 1100us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:34.721202 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:34.721206 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:34.721386 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:34.722074 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:34.722082 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:34.722089 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:34.722514 (+ 425us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:34.722693 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:34.723898 (+ 1205us) server_negotiation.cc:299] Negotiation successful
0809 12:16:34.724203 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 12:16:34.725214 23196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:34.699638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45709 (local address 127.19.183.195:42035)
0809 12:16:34.700635 (+ 997us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:34.700670 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:34.702490 (+ 1820us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:34.703926 (+ 1436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:34.703940 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:34.704796 (+ 856us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:34.705940 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:34.705972 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.707539 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.707547 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:34.709633 (+ 2086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:34.709646 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.710823 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.710832 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:34.711037 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:34.712045 (+ 1008us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:34.712095 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:34.715506 (+ 3411us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:34.720363 (+ 4857us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:34.720375 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:34.720396 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:34.721029 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:34.721574 (+ 545us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:34.721580 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:34.721586 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:34.721896 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:34.722818 (+ 922us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:34.722831 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:34.723251 (+ 420us) client_negotiation.cc:769] Sending connection context
0809 12:16:34.723698 (+ 447us) client_negotiation.cc:240] Negotiation successful
0809 12:16:34.724081 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"thread_start_us":435,"threads_started":1}
I20260809 12:16:34.747790 23298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:34.732610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45709 (local address 127.19.183.195:44269)
0809 12:16:34.733231 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:34.733245 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:34.733423 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:34.733755 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:34.733760 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:34.734461 (+ 701us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:34.734959 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:34.734976 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.736331 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.736337 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:34.738186 (+ 1849us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:34.738201 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.739051 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.739059 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:34.739237 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:34.740086 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:34.740114 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:34.741337 (+ 1223us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:34.744702 (+ 3365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:34.744707 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:34.744711 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:34.745117 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:34.745549 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:34.745554 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:34.745557 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:34.745756 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:34.746573 (+ 817us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:34.746581 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:34.746863 (+ 282us) client_negotiation.cc:769] Sending connection context
0809 12:16:34.747003 (+ 140us) client_negotiation.cc:240] Negotiation successful
0809 12:16:34.747293 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":684,"threads_started":1}
I20260809 12:16:34.747830 22822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:34.732465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:44269 (local address 127.19.183.254:45709)
0809 12:16:34.732670 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:34.732676 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:34.733356 (+ 680us) server_negotiation.cc:407] Connection header received
0809 12:16:34.733550 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:34.733555 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:34.733627 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:34.733725 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:34.735182 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.736200 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.738352 (+ 2152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.738860 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.738990 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:34.741591 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:34.741598 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:34.741603 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:34.741644 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:34.744569 (+ 2925us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:34.745243 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:34.745249 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:34.745253 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:34.745376 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:34.745965 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:34.745971 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:34.745974 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:34.746420 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:34.746619 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:34.747134 (+ 515us) server_negotiation.cc:299] Negotiation successful
0809 12:16:34.747383 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 12:16:34.749221 23164 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:45405
I20260809 12:16:34.749315 23346 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:45405 every 8 connection(s)
I20260809 12:16:34.750618 23164 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:16:34.751120 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 23164
I20260809 12:16:34.751302 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:16:34.752736 20191 external_mini_cluster.cc:1428] Setting key a2f1d712ecaa3f88cf50da4efdd3e724
I20260809 12:16:34.752985 23347 heartbeater.cc:347] Connected to a master server at 127.19.183.254:45709
I20260809 12:16:34.753680 23347 heartbeater.cc:464] Registering TS with master...
I20260809 12:16:34.754436 23347 heartbeater.cc:511] Master 127.19.183.254:45709 requested a full tablet report, sending...
I20260809 12:16:34.755913 22682 ts_manager.cc:194] Registered new tserver with Master: 4f3869b285954006b6ddf897a37b918f (127.19.183.195:45405)
I20260809 12:16:34.756718 22682 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:42035
I20260809 12:16:34.767820 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 12:16:34 dist-test-slave-39l8 krb5kdc[22631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 12:16:34 dist-test-slave-39l8 krb5kdc[22631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277794, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:34.822623 22822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:34.808848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56652 (local address 127.19.183.254:45709)
0809 12:16:34.809078 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:34.809084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:34.809249 (+ 165us) server_negotiation.cc:407] Connection header received
0809 12:16:34.809474 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:34.809478 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:34.809544 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:34.809657 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:34.811193 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.811938 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.813240 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.813705 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.813840 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:34.817586 (+ 3746us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:34.817591 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:34.817599 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:34.817632 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:34.820072 (+ 2440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:34.820719 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:34.820726 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:34.820729 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:34.820859 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:34.821222 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:34.821227 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:34.821231 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:34.821492 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:34.821761 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:34.822041 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 12:16:34.822265 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 12:16:34.826421 22682 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56652:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:16:34.833521 22682 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:16:34.857968 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:34.844874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:37075 (local address 127.0.0.1:42514)
0809 12:16:34.845820 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:34.845868 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:34.846077 (+ 209us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:34.847448 (+ 1371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:34.847454 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:34.847485 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:34.847942 (+ 457us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:34.847962 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.850637 (+ 2675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.850646 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:34.853888 (+ 3242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:34.853911 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.856647 (+ 2736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.856657 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:34.856959 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:34.857031 (+ 72us) client_negotiation.cc:769] Sending connection context
0809 12:16:34.857186 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 12:16:34.857434 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":1494,"threads_started":1}
I20260809 12:16:34.860154 23366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:34.845107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42514 (local address 127.19.183.193:37075)
0809 12:16:34.846779 (+ 1672us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:34.846790 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:34.846824 (+ 34us) server_negotiation.cc:407] Connection header received
0809 12:16:34.846931 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:34.846940 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:34.847196 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:34.848226 (+ 1030us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:34.848473 (+ 247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.850427 (+ 1954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.854250 (+ 3823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.856328 (+ 2078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.859038 (+ 2710us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:34.859286 (+ 248us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:34.859371 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 12:16:34.859624 (+ 253us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1472,"thread_start_us":1344,"threads_started":1}
I20260809 12:16:34.865423 22880 tablet_service.cc:1467] Processing CreateTablet for tablet e1feedd4cdda4a56b6fe59a4da0948ab (DEFAULT_TABLE table=test-table [id=12e2b142eca349809724efa5b33e11e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:34.866899 22880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1feedd4cdda4a56b6fe59a4da0948ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:34.867214 23365 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:34.845046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:45405 (local address 127.0.0.1:38242)
0809 12:16:34.847114 (+ 2068us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:34.847133 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:34.847245 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:34.859838 (+ 12593us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:34.859846 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:34.859886 (+ 40us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:34.860352 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:34.860367 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.862634 (+ 2267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.862639 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:34.864528 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:34.864544 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.866369 (+ 1825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.866376 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:34.866529 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:34.866569 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 12:16:34.866694 (+ 125us) client_negotiation.cc:240] Negotiation successful
0809 12:16:34.866845 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1942,"mutex_wait_us":1,"thread_start_us":1771,"threads_started":1}
I20260809 12:16:34.867250 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:34.845641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38242 (local address 127.19.183.195:45405)
0809 12:16:34.858810 (+ 13169us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:34.858824 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:34.858866 (+ 42us) server_negotiation.cc:407] Connection header received
0809 12:16:34.858985 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:34.858996 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:34.859280 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:34.860596 (+ 1316us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:34.860922 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.862505 (+ 1583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.864723 (+ 2218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.866153 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.866372 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:34.866533 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:34.866777 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 12:16:34.866921 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12887,"thread_start_us":13819,"threads_started":1}
I20260809 12:16:34.868429 23364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:34.847209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:42423 (local address 127.0.0.1:37078)
0809 12:16:34.858275 (+ 11066us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:34.858299 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:34.858423 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:34.861637 (+ 3214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:34.861642 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:34.861669 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:34.861844 (+ 175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:34.861853 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.863826 (+ 1973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.863832 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:34.865753 (+ 1921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:34.865768 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.867726 (+ 1958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.867732 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:34.867948 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:34.867979 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 12:16:34.868050 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 12:16:34.868188 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10929,"thread_start_us":13457,"threads_started":1}
I20260809 12:16:34.869356 23369 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:34.847704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37078 (local address 127.19.183.194:42423)
0809 12:16:34.861089 (+ 13385us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:34.861095 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:34.861119 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:16:34.861191 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:34.861196 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:34.861398 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:34.862000 (+ 602us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:34.862200 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.863622 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.865929 (+ 2307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:34.867304 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:34.868714 (+ 1410us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:34.868880 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:34.868952 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 12:16:34.869100 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13234,"thread_start_us":13129,"threads_started":1}
I20260809 12:16:34.870975 23252 tablet_service.cc:1467] Processing CreateTablet for tablet e1feedd4cdda4a56b6fe59a4da0948ab (DEFAULT_TABLE table=test-table [id=12e2b142eca349809724efa5b33e11e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:34.872833 23252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1feedd4cdda4a56b6fe59a4da0948ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:34.872795 23061 tablet_service.cc:1467] Processing CreateTablet for tablet e1feedd4cdda4a56b6fe59a4da0948ab (DEFAULT_TABLE table=test-table [id=12e2b142eca349809724efa5b33e11e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:34.874102 23061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e1feedd4cdda4a56b6fe59a4da0948ab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:34.879523 23370 tablet_bootstrap.cc:492] T e1feedd4cdda4a56b6fe59a4da0948ab P dfc9eb51ad9848f08a94c57a60c7c1c5: Bootstrap starting.
I20260809 12:16:34.883133 23370 tablet_bootstrap.cc:654] T e1feedd4cdda4a56b6fe59a4da0948ab P dfc9eb51ad9848f08a94c57a60c7c1c5: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:34.884333 23370 log.cc:824] T e1feedd4cdda4a56b6fe59a4da0948ab P dfc9eb51ad9848f08a94c57a60c7c1c5: Log is configured to *not* fsync() on all Append() calls
I20260809 12:16:34.887020 23370 tablet_bootstrap.cc:492] T e1feedd4cdda4a56b6fe59a4da0948ab P dfc9eb51ad9848f08a94c57a60c7c1c5: No bootstrap required, opened a new log
I20260809 12:16:34.887357 23370 ts_tablet_manager.cc:1397] T e1feedd4cdda4a56b6fe59a4da0948ab P dfc9eb51ad9848f08a94c57a60c7c1c5: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.005s
I20260809 12:16:34.887960 23373 tablet_bootstrap.cc:492] T e1feedd4cdda4a56b6fe59a4da0948ab P 4f3869b285954006b6ddf897a37b918f: Bootstrap starting.
I20260809 12:16:34.891892 23373 tablet_bootstrap.cc:654] T e1feedd4cdda4a56b6fe59a4da0948ab P 4f3869b285954006b6ddf897a37b918f: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:34.893149 23373 log.cc:824] T e1feedd4cdda4a56b6fe59a4da0948ab P 4f3869b285954006b6ddf897a37b918f: Log is configured to *not* fsync() on all Append() calls
I20260809 12:16:34.893947 23374 tablet_bootstrap.cc:492] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd: Bootstrap starting.
I20260809 12:16:34.896821 23374 tablet_bootstrap.cc:654] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:34.896920 23373 tablet_bootstrap.cc:492] T e1feedd4cdda4a56b6fe59a4da0948ab P 4f3869b285954006b6ddf897a37b918f: No bootstrap required, opened a new log
I20260809 12:16:34.897207 23373 ts_tablet_manager.cc:1397] T e1feedd4cdda4a56b6fe59a4da0948ab P 4f3869b285954006b6ddf897a37b918f: Time spent bootstrapping tablet: real 0.010s user 0.007s sys 0.000s
I20260809 12:16:34.897854 23374 log.cc:824] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd: Log is configured to *not* fsync() on all Append() calls
I20260809 12:16:34.898464 23370 raft_consensus.cc:348] T e1feedd4cdda4a56b6fe59a4da0948ab P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } }
I20260809 12:16:34.899047 23370 raft_consensus.cc:374] T e1feedd4cdda4a56b6fe59a4da0948ab P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:34.899190 23370 raft_consensus.cc:729] T e1feedd4cdda4a56b6fe59a4da0948ab P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfc9eb51ad9848f08a94c57a60c7c1c5, State: Initialized, Role: FOLLOWER
I20260809 12:16:34.899977 23370 consensus_queue.cc:260] T e1feedd4cdda4a56b6fe59a4da0948ab P dfc9eb51ad9848f08a94c57a60c7c1c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } }
I20260809 12:16:34.901029 23374 tablet_bootstrap.cc:492] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd: No bootstrap required, opened a new log
I20260809 12:16:34.901336 23374 ts_tablet_manager.cc:1397] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 12:16:34.901777 23370 ts_tablet_manager.cc:1428] T e1feedd4cdda4a56b6fe59a4da0948ab P dfc9eb51ad9848f08a94c57a60c7c1c5: Time spent starting tablet: real 0.014s user 0.009s sys 0.003s
I20260809 12:16:34.907457 23373 raft_consensus.cc:348] T e1feedd4cdda4a56b6fe59a4da0948ab P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } }
I20260809 12:16:34.907833 23373 raft_consensus.cc:374] T e1feedd4cdda4a56b6fe59a4da0948ab P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:34.907941 23373 raft_consensus.cc:729] T e1feedd4cdda4a56b6fe59a4da0948ab P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f3869b285954006b6ddf897a37b918f, State: Initialized, Role: FOLLOWER
I20260809 12:16:34.908459 23373 consensus_queue.cc:260] T e1feedd4cdda4a56b6fe59a4da0948ab P 4f3869b285954006b6ddf897a37b918f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } }
I20260809 12:16:34.910069 23373 ts_tablet_manager.cc:1428] T e1feedd4cdda4a56b6fe59a4da0948ab P 4f3869b285954006b6ddf897a37b918f: Time spent starting tablet: real 0.013s user 0.014s sys 0.000s
I20260809 12:16:34.910166 23347 heartbeater.cc:503] Master 127.19.183.254:45709 was elected leader, sending a full tablet report...
I20260809 12:16:34.909896 23374 raft_consensus.cc:348] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } }
I20260809 12:16:34.910315 23374 raft_consensus.cc:374] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:34.910431 23374 raft_consensus.cc:729] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6183387a03104ef6abd3836001c0dcdd, State: Initialized, Role: FOLLOWER
I20260809 12:16:34.911193 23374 consensus_queue.cc:260] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } }
I20260809 12:16:34.912351 23160 heartbeater.cc:503] Master 127.19.183.254:45709 was elected leader, sending a full tablet report...
I20260809 12:16:34.912954 23374 ts_tablet_manager.cc:1428] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd: Time spent starting tablet: real 0.011s user 0.003s sys 0.007s
W20260809 12:16:34.946460 22974 tablet.cc:2369] T e1feedd4cdda4a56b6fe59a4da0948ab P dfc9eb51ad9848f08a94c57a60c7c1c5: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:16:34.976207 23161 tablet.cc:2369] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:16:35.003000 23349 tablet.cc:2369] T e1feedd4cdda4a56b6fe59a4da0948ab P 4f3869b285954006b6ddf897a37b918f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:16:35.111919 23381 raft_consensus.cc:482] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:16:35.112287 23381 raft_consensus.cc:504] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } }
I20260809 12:16:35.114078 23381 leader_election.cc:288] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dfc9eb51ad9848f08a94c57a60c7c1c5 (127.19.183.193:37075), 4f3869b285954006b6ddf897a37b918f (127.19.183.195:45405)
I20260809 12:16:35.121039 23366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:35.114414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:41033 (local address 127.19.183.193:37075)
0809 12:16:35.114665 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:35.114672 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:35.115587 (+ 915us) server_negotiation.cc:407] Connection header received
0809 12:16:35.115787 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:35.115792 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:35.115854 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:35.115953 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:35.116625 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.117344 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.119306 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.120069 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.120278 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:35.120358 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:35.120659 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 12:16:35.120790 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 12:16:35.121219 23383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:35.114537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:37075 (local address 127.19.183.194:41033)
0809 12:16:35.115465 (+ 928us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:35.115481 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:35.115614 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:35.116038 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:35.116046 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:35.116083 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:35.116470 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:35.116480 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.117480 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.117488 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:35.119133 (+ 1645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:35.119146 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.120314 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.120320 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:35.120476 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:35.120501 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 12:16:35.120629 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 12:16:35.120759 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"mutex_wait_us":21,"thread_start_us":585,"threads_started":1}
I20260809 12:16:35.121807 23384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:35.114586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:45405 (local address 127.19.183.194:54313)
0809 12:16:35.116168 (+ 1582us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:35.116182 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:35.116355 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:35.116706 (+ 351us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:35.116711 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:35.116737 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:35.117066 (+ 329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:35.117075 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.118099 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.118111 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:35.119714 (+ 1603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:35.119728 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.120982 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.120988 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:35.121190 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:35.121236 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 12:16:35.121360 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 12:16:35.121500 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1446,"mutex_wait_us":1,"thread_start_us":1215,"threads_started":1}
I20260809 12:16:35.121857 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:35.114769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:54313 (local address 127.19.183.195:45405)
0809 12:16:35.115015 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:35.115024 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:35.116282 (+ 1258us) server_negotiation.cc:407] Connection header received
0809 12:16:35.116507 (+ 225us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:35.116517 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:35.116578 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:35.116683 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:35.117238 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.117937 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.119981 (+ 2044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.120754 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.120950 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:35.121025 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:35.121442 (+ 417us) server_negotiation.cc:299] Negotiation successful
0809 12:16:35.121609 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 12:16:35.122393 22912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1feedd4cdda4a56b6fe59a4da0948ab" candidate_uuid: "6183387a03104ef6abd3836001c0dcdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" is_pre_election: true
I20260809 12:16:35.123081 22912 raft_consensus.cc:2454] T e1feedd4cdda4a56b6fe59a4da0948ab P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6183387a03104ef6abd3836001c0dcdd in term 0.
I20260809 12:16:35.122973 23279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1feedd4cdda4a56b6fe59a4da0948ab" candidate_uuid: "6183387a03104ef6abd3836001c0dcdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f3869b285954006b6ddf897a37b918f" is_pre_election: true
I20260809 12:16:35.123571 23279 raft_consensus.cc:2454] T e1feedd4cdda4a56b6fe59a4da0948ab P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6183387a03104ef6abd3836001c0dcdd in term 0.
I20260809 12:16:35.123826 23002 leader_election.cc:302] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6183387a03104ef6abd3836001c0dcdd, dfc9eb51ad9848f08a94c57a60c7c1c5; no voters:
I20260809 12:16:35.124251 23381 raft_consensus.cc:2781] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:16:35.124402 23381 raft_consensus.cc:482] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:16:35.124516 23381 raft_consensus.cc:3037] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:35.126660 23381 raft_consensus.cc:504] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } }
I20260809 12:16:35.127395 23381 leader_election.cc:288] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd [CANDIDATE]: Term 1 election: Requested vote from peers dfc9eb51ad9848f08a94c57a60c7c1c5 (127.19.183.193:37075), 4f3869b285954006b6ddf897a37b918f (127.19.183.195:45405)
I20260809 12:16:35.127808 22912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1feedd4cdda4a56b6fe59a4da0948ab" candidate_uuid: "6183387a03104ef6abd3836001c0dcdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5"
I20260809 12:16:35.127931 23279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e1feedd4cdda4a56b6fe59a4da0948ab" candidate_uuid: "6183387a03104ef6abd3836001c0dcdd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f3869b285954006b6ddf897a37b918f"
I20260809 12:16:35.128036 22912 raft_consensus.cc:3037] T e1feedd4cdda4a56b6fe59a4da0948ab P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:35.128153 23279 raft_consensus.cc:3037] T e1feedd4cdda4a56b6fe59a4da0948ab P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:35.131426 22912 raft_consensus.cc:2454] T e1feedd4cdda4a56b6fe59a4da0948ab P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6183387a03104ef6abd3836001c0dcdd in term 1.
I20260809 12:16:35.131426 23279 raft_consensus.cc:2454] T e1feedd4cdda4a56b6fe59a4da0948ab P 4f3869b285954006b6ddf897a37b918f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6183387a03104ef6abd3836001c0dcdd in term 1.
I20260809 12:16:35.131857 23002 leader_election.cc:302] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6183387a03104ef6abd3836001c0dcdd, dfc9eb51ad9848f08a94c57a60c7c1c5; no voters:
I20260809 12:16:35.132220 23381 raft_consensus.cc:2781] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:16:35.132866 23381 raft_consensus.cc:686] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd [term 1 LEADER]: Becoming Leader. State: Replica: 6183387a03104ef6abd3836001c0dcdd, State: Running, Role: LEADER
I20260809 12:16:35.133401 23381 consensus_queue.cc:237] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } }
I20260809 12:16:35.138320 22681 catalog_manager.cc:5495] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd reported cstate change: term changed from 0 to 1, leader changed from <none> to 6183387a03104ef6abd3836001c0dcdd (127.19.183.194). New cstate: current_term: 1 leader_uuid: "6183387a03104ef6abd3836001c0dcdd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } health_report { overall_health: HEALTHY } } }
Aug 09 12:16:35 dist-test-slave-39l8 krb5kdc[22631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277792, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:35.202674 22822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:35.189238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56662 (local address 127.19.183.254:45709)
0809 12:16:35.189460 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:35.189469 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:35.189987 (+ 518us) server_negotiation.cc:407] Connection header received
0809 12:16:35.190125 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:35.190129 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:35.190215 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:35.190321 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:35.191568 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.192344 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.193527 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.193934 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.194024 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:35.197259 (+ 3235us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:35.197264 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:35.197267 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:35.197296 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:35.199894 (+ 2598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:35.200590 (+ 696us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:35.200604 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:35.200609 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:35.200733 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:35.201092 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:35.201097 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:35.201102 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:35.201372 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:35.201524 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:35.201939 (+ 415us) server_negotiation.cc:299] Negotiation successful
0809 12:16:35.202293 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":170,"server-negotiator.queue_time_us":68}
I20260809 12:16:35.202732 23394 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:35.189257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45709 (local address 127.0.0.1:56662)
0809 12:16:35.189878 (+ 621us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:35.189897 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:35.190020 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:35.190378 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:35.190382 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:35.191143 (+ 761us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:35.191433 (+ 290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:35.191443 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.192468 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.192472 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:35.193406 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:35.193417 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.194091 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.194095 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:35.194217 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:35.194889 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:35.194906 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:35.197064 (+ 2158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:35.200029 (+ 2965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:35.200044 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:35.200051 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:35.200469 (+ 418us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:35.200853 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:35.200858 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:35.200862 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:35.200988 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:35.201504 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:35.201512 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:35.201693 (+ 181us) client_negotiation.cc:769] Sending connection context
0809 12:16:35.201872 (+ 179us) client_negotiation.cc:240] Negotiation successful
0809 12:16:35.202181 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"mutex_wait_us":1,"thread_start_us":342,"threads_started":1}
I20260809 12:16:35.207811 22681 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:56662:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 12:16:35.208786 22681 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:16:35.215415 22880 tablet_service.cc:1467] Processing CreateTablet for tablet 7e8079d51d644212902ea5a305289571 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5ba3ee5cdfbe4a1e85514df45acfb115]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:16:35.215847 23252 tablet_service.cc:1467] Processing CreateTablet for tablet 7e8079d51d644212902ea5a305289571 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5ba3ee5cdfbe4a1e85514df45acfb115]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:16:35.216475 22880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e8079d51d644212902ea5a305289571. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:35.216647 23252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e8079d51d644212902ea5a305289571. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:35.219261 23061 tablet_service.cc:1467] Processing CreateTablet for tablet 7e8079d51d644212902ea5a305289571 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=5ba3ee5cdfbe4a1e85514df45acfb115]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:16:35.220121 23061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7e8079d51d644212902ea5a305289571. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:35.223968 23373 tablet_bootstrap.cc:492] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f: Bootstrap starting.
I20260809 12:16:35.223968 23370 tablet_bootstrap.cc:492] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5: Bootstrap starting.
I20260809 12:16:35.225682 23373 tablet_bootstrap.cc:654] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:35.225690 23370 tablet_bootstrap.cc:654] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:35.227445 23374 tablet_bootstrap.cc:492] T 7e8079d51d644212902ea5a305289571 P 6183387a03104ef6abd3836001c0dcdd: Bootstrap starting.
I20260809 12:16:35.229209 23374 tablet_bootstrap.cc:654] T 7e8079d51d644212902ea5a305289571 P 6183387a03104ef6abd3836001c0dcdd: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:35.236990 23373 tablet_bootstrap.cc:492] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f: No bootstrap required, opened a new log
I20260809 12:16:35.237231 23373 ts_tablet_manager.cc:1397] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.011s
I20260809 12:16:35.237480 23370 tablet_bootstrap.cc:492] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5: No bootstrap required, opened a new log
I20260809 12:16:35.237641 23370 ts_tablet_manager.cc:1397] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5: Time spent bootstrapping tablet: real 0.014s user 0.001s sys 0.009s
I20260809 12:16:35.238276 23373 raft_consensus.cc:348] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:35.238384 23370 raft_consensus.cc:348] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:35.238613 23370 raft_consensus.cc:374] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:35.238688 23373 raft_consensus.cc:374] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:35.238760 23370 raft_consensus.cc:729] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfc9eb51ad9848f08a94c57a60c7c1c5, State: Initialized, Role: FOLLOWER
I20260809 12:16:35.238971 23373 raft_consensus.cc:729] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f3869b285954006b6ddf897a37b918f, State: Initialized, Role: FOLLOWER
I20260809 12:16:35.239015 23370 consensus_queue.cc:260] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:35.239709 23370 ts_tablet_manager.cc:1428] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 12:16:35.239573 23377 tablet_replica.cc:439] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } } }
I20260809 12:16:35.240032 23377 tablet_replica.cc:442] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:16:35.239377 23373 consensus_queue.cc:260] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:35.240720 23380 tablet_replica.cc:439] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } } }
I20260809 12:16:35.241015 23380 tablet_replica.cc:442] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:16:35.241525 23374 tablet_bootstrap.cc:492] T 7e8079d51d644212902ea5a305289571 P 6183387a03104ef6abd3836001c0dcdd: No bootstrap required, opened a new log
I20260809 12:16:35.241673 23374 ts_tablet_manager.cc:1397] T 7e8079d51d644212902ea5a305289571 P 6183387a03104ef6abd3836001c0dcdd: Time spent bootstrapping tablet: real 0.014s user 0.008s sys 0.004s
I20260809 12:16:35.242542 23374 raft_consensus.cc:348] T 7e8079d51d644212902ea5a305289571 P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:35.242861 23374 raft_consensus.cc:374] T 7e8079d51d644212902ea5a305289571 P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:35.242962 23374 raft_consensus.cc:729] T 7e8079d51d644212902ea5a305289571 P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6183387a03104ef6abd3836001c0dcdd, State: Initialized, Role: FOLLOWER
I20260809 12:16:35.243264 23374 consensus_queue.cc:260] T 7e8079d51d644212902ea5a305289571 P 6183387a03104ef6abd3836001c0dcdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:35.243677 23381 tablet_replica.cc:439] T 7e8079d51d644212902ea5a305289571 P 6183387a03104ef6abd3836001c0dcdd: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } } }
I20260809 12:16:35.243947 23381 tablet_replica.cc:442] T 7e8079d51d644212902ea5a305289571 P 6183387a03104ef6abd3836001c0dcdd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:16:35.244462 23374 ts_tablet_manager.cc:1428] T 7e8079d51d644212902ea5a305289571 P 6183387a03104ef6abd3836001c0dcdd: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 12:16:35.246215 23373 ts_tablet_manager.cc:1428] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f: Time spent starting tablet: real 0.009s user 0.000s sys 0.002s
I20260809 12:16:35.580654 23380 raft_consensus.cc:482] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:16:35.580893 23380 raft_consensus.cc:504] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:35.581290 23377 raft_consensus.cc:482] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:16:35.581499 23377 raft_consensus.cc:504] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:35.582644 23380 leader_election.cc:288] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers dfc9eb51ad9848f08a94c57a60c7c1c5 (127.19.183.193:37075), 6183387a03104ef6abd3836001c0dcdd (127.19.183.194:42423)
I20260809 12:16:35.583144 23377 leader_election.cc:288] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6183387a03104ef6abd3836001c0dcdd (127.19.183.194:42423), 4f3869b285954006b6ddf897a37b918f (127.19.183.195:45405)
I20260809 12:16:35.587327 23381 consensus_queue.cc:1035] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:16:35.591444 23366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:35.583243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:57203 (local address 127.19.183.193:37075)
0809 12:16:35.583439 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:35.583446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:35.583996 (+ 550us) server_negotiation.cc:407] Connection header received
0809 12:16:35.584318 (+ 322us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:35.584326 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:35.584407 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:35.584476 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:35.585180 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.586315 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.588728 (+ 2413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.589910 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.590083 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:35.590194 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:35.590872 (+ 678us) server_negotiation.cc:299] Negotiation successful
0809 12:16:35.591051 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 12:16:35.592430 23403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:35.583017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:37075 (local address 127.19.183.195:57203)
0809 12:16:35.583874 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:35.583901 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:35.584125 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:35.584700 (+ 575us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:35.584705 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:35.584733 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:35.584967 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:35.584979 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.586532 (+ 1553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.586536 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:35.588527 (+ 1991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:35.588543 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.590357 (+ 1814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.590366 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:35.590632 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:35.590684 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 12:16:35.591816 (+ 1132us) client_negotiation.cc:240] Negotiation successful
0809 12:16:35.592034 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":740,"thread_start_us":3081,"threads_started":1}
I20260809 12:16:35.599232 22911 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e8079d51d644212902ea5a305289571" candidate_uuid: "4f3869b285954006b6ddf897a37b918f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" is_pre_election: true
I20260809 12:16:35.599970 23385 consensus_queue.cc:1035] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:16:35.600068 22911 raft_consensus.cc:2454] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f3869b285954006b6ddf897a37b918f in term 0.
I20260809 12:16:35.601110 23190 leader_election.cc:302] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f3869b285954006b6ddf897a37b918f, dfc9eb51ad9848f08a94c57a60c7c1c5; no voters:
I20260809 12:16:35.602932 23380 raft_consensus.cc:2781] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:16:35.603516 23380 raft_consensus.cc:482] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:16:35.603673 23380 raft_consensus.cc:3037] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:35.609699 23380 raft_consensus.cc:504] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:35.610805 23380 leader_election.cc:288] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f [CANDIDATE]: Term 1 election: Requested vote from peers dfc9eb51ad9848f08a94c57a60c7c1c5 (127.19.183.193:37075), 6183387a03104ef6abd3836001c0dcdd (127.19.183.194:42423)
I20260809 12:16:35.611403 22912 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e8079d51d644212902ea5a305289571" candidate_uuid: "4f3869b285954006b6ddf897a37b918f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5"
I20260809 12:16:35.611800 22912 raft_consensus.cc:3037] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:35.616603 22912 raft_consensus.cc:2454] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f3869b285954006b6ddf897a37b918f in term 1.
I20260809 12:16:35.618511 23409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:35.586003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:34345 (local address 127.19.183.194:42423)
0809 12:16:35.593719 (+ 7716us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:35.593725 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:35.598055 (+ 4330us) server_negotiation.cc:407] Connection header received
0809 12:16:35.598553 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:35.598559 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:35.598638 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:35.598753 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:35.602542 (+ 3789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.603720 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.608269 (+ 4549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.610119 (+ 1850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.610335 (+ 216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:35.610478 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:35.617795 (+ 7317us) server_negotiation.cc:299] Negotiation successful
0809 12:16:35.618022 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7600,"thread_start_us":8002,"threads_started":1}
I20260809 12:16:35.619114 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:35.584343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:59875 (local address 127.19.183.195:45405)
0809 12:16:35.584507 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:35.584512 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:35.597735 (+ 13223us) server_negotiation.cc:407] Connection header received
0809 12:16:35.598865 (+ 1130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:35.598871 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:35.598945 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:35.599000 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:35.602534 (+ 3534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.607529 (+ 4995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.615676 (+ 8147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.617431 (+ 1755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.617608 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:35.617714 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:35.618500 (+ 786us) server_negotiation.cc:299] Negotiation successful
0809 12:16:35.618732 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 12:16:35.619249 23190 leader_election.cc:302] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f3869b285954006b6ddf897a37b918f, dfc9eb51ad9848f08a94c57a60c7c1c5; no voters:
I20260809 12:16:35.620645 23380 raft_consensus.cc:2781] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:16:35.621142 23405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:35.583556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:42423 (local address 127.19.183.193:34345)
0809 12:16:35.597951 (+ 14395us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:35.597967 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:35.598354 (+ 387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:35.598962 (+ 608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:35.598968 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:35.598995 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:35.599601 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:35.599618 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.603927 (+ 4309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.603936 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:35.608014 (+ 4078us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:35.608036 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.616833 (+ 8797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.616848 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:35.617108 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:35.617167 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 12:16:35.617281 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 12:16:35.617491 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14284,"thread_start_us":14197,"threads_started":1}
I20260809 12:16:35.621214 23407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:35.583422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:45405 (local address 127.19.183.193:59875)
0809 12:16:35.597570 (+ 14148us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:35.597611 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:35.598672 (+ 1061us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:35.599217 (+ 545us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:35.599221 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:35.599242 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:35.602338 (+ 3096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:35.602354 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.613228 (+ 10874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.613236 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:35.615387 (+ 2151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:35.615401 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.617958 (+ 2557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.617968 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:35.618230 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:35.618281 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 12:16:35.619317 (+ 1036us) client_negotiation.cc:240] Negotiation successful
0809 12:16:35.619667 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13946,"mutex_wait_us":4,"thread_start_us":14653,"threads_started":1}
I20260809 12:16:35.622253 23278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e8079d51d644212902ea5a305289571" candidate_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f3869b285954006b6ddf897a37b918f" is_pre_election: true
I20260809 12:16:35.623379 23087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e8079d51d644212902ea5a305289571" candidate_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6183387a03104ef6abd3836001c0dcdd" is_pre_election: true
I20260809 12:16:35.623709 23087 raft_consensus.cc:2454] T 7e8079d51d644212902ea5a305289571 P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfc9eb51ad9848f08a94c57a60c7c1c5 in term 0.
I20260809 12:16:35.624533 22817 leader_election.cc:302] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6183387a03104ef6abd3836001c0dcdd, dfc9eb51ad9848f08a94c57a60c7c1c5; no voters:
I20260809 12:16:35.624910 23377 raft_consensus.cc:2741] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 1 FOLLOWER]: Leader pre-election decision vote started in defunct term 0: won
I20260809 12:16:35.626565 23380 raft_consensus.cc:686] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f [term 1 LEADER]: Becoming Leader. State: Replica: 4f3869b285954006b6ddf897a37b918f, State: Running, Role: LEADER
I20260809 12:16:35.627167 23403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:35.586519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:42423 (local address 127.19.183.195:49313)
0809 12:16:35.592901 (+ 6382us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:35.592925 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:35.593041 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:35.621679 (+ 28638us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:35.621684 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:35.621720 (+ 36us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:35.621941 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:35.621954 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.623425 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.623430 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:35.625178 (+ 1748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:35.625193 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.626207 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.626212 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:35.626480 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:35.626526 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 12:16:35.626643 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 12:16:35.626788 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6254,"thread_start_us":7223,"threads_started":1}
I20260809 12:16:35.627245 23409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:35.594186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:49313 (local address 127.19.183.194:42423)
0809 12:16:35.620205 (+ 26019us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:35.620212 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:35.620232 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:16:35.620309 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:35.620314 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:35.620384 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:35.620521 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:35.622275 (+ 1754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.623245 (+ 970us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.625375 (+ 2130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.626031 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.626195 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:35.626270 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:35.626849 (+ 579us) server_negotiation.cc:299] Negotiation successful
0809 12:16:35.626969 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25912,"thread_start_us":25850,"threads_started":1}
I20260809 12:16:35.627131 23380 consensus_queue.cc:237] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:35.627825 23087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e8079d51d644212902ea5a305289571" candidate_uuid: "4f3869b285954006b6ddf897a37b918f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6183387a03104ef6abd3836001c0dcdd" is_pre_election: true
I20260809 12:16:35.627979 23085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7e8079d51d644212902ea5a305289571" candidate_uuid: "4f3869b285954006b6ddf897a37b918f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6183387a03104ef6abd3836001c0dcdd"
I20260809 12:16:35.628084 23087 raft_consensus.cc:2454] T 7e8079d51d644212902ea5a305289571 P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f3869b285954006b6ddf897a37b918f in term 0.
I20260809 12:16:35.631244 23422 tablet_replica.cc:439] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f: TxnStatusTablet state changed. Reason: New leader 4f3869b285954006b6ddf897a37b918f. Latest consensus state: current_term: 1 leader_uuid: "4f3869b285954006b6ddf897a37b918f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } } }
I20260809 12:16:35.631598 23422 tablet_replica.cc:442] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:16:35.632680 23425 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 12:16:35.632973 23425 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 12:16:35.634222 22682 catalog_manager.cc:5495] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f3869b285954006b6ddf897a37b918f (127.19.183.195). New cstate: current_term: 1 leader_uuid: "4f3869b285954006b6ddf897a37b918f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } health_report { overall_health: HEALTHY } } }
I20260809 12:16:35.712307 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:35.704277 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38256 (local address 127.19.183.195:45405)
0809 12:16:35.704470 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:35.704476 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:35.704990 (+ 514us) server_negotiation.cc:407] Connection header received
0809 12:16:35.705311 (+ 321us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:35.705316 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:35.705413 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:35.705589 (+ 176us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:16:35.706018 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.707251 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.709269 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.710004 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.710126 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:35.710741 (+ 615us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:16:35.711517 (+ 776us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:16:35.711759 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:35.711872 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 12:16:35.711969 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 12:16:35.712443 23428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:35.704151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:45405 (local address 127.0.0.1:38256)
0809 12:16:35.704873 (+ 722us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:35.704892 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:35.705053 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:35.705644 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:35.705649 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:35.705662 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:16:35.705875 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:35.705886 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.707410 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.707416 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:35.709090 (+ 1674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:35.709107 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.710179 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.710186 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:35.710364 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:35.710440 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:16:35.711728 (+ 1288us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:16:35.711751 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 12:16:35.711844 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 12:16:35.712055 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":558,"thread_start_us":433,"threads_started":1}
I20260809 12:16:35.720064 23085 raft_consensus.cc:3037] T 7e8079d51d644212902ea5a305289571 P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:35.720525 22912 raft_consensus.cc:1261] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 1 FOLLOWER]: Refusing update from remote peer 4f3869b285954006b6ddf897a37b918f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:16:35.721253 23422 consensus_queue.cc:1035] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f [LEADER]: Connected to new peer: Peer: permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:16:35.723336 23085 raft_consensus.cc:1261] T 7e8079d51d644212902ea5a305289571 P 6183387a03104ef6abd3836001c0dcdd [term 1 FOLLOWER]: Refusing update from remote peer 4f3869b285954006b6ddf897a37b918f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:16:35.724079 23422 consensus_queue.cc:1035] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f [LEADER]: Connected to new peer: Peer: permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:16:35.732625 23377 tablet_replica.cc:439] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5: TxnStatusTablet state changed. Reason: New leader 4f3869b285954006b6ddf897a37b918f. Latest consensus state: current_term: 1 leader_uuid: "4f3869b285954006b6ddf897a37b918f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } } }
I20260809 12:16:35.732952 23377 tablet_replica.cc:442] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:16:35.734608 23420 mvcc.cc:205] Tried to move back new op lower bound from 7316593851260370944 to 7316593850895601664. Current Snapshot: MvccSnapshot[applied={T|T < 7316593851260370944}]
I20260809 12:16:35.738265 23385 tablet_replica.cc:439] T 7e8079d51d644212902ea5a305289571 P 6183387a03104ef6abd3836001c0dcdd: TxnStatusTablet state changed. Reason: New leader 4f3869b285954006b6ddf897a37b918f. Latest consensus state: current_term: 1 leader_uuid: "4f3869b285954006b6ddf897a37b918f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } } }
I20260809 12:16:35.738585 23385 tablet_replica.cc:442] T 7e8079d51d644212902ea5a305289571 P 6183387a03104ef6abd3836001c0dcdd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:16:35.740957 23416 mvcc.cc:205] Tried to move back new op lower bound from 7316593851260370944 to 7316593850895601664. Current Snapshot: MvccSnapshot[applied={T|T < 7316593851260370944}]
I20260809 12:16:35.742012 23385 tablet_replica.cc:439] T 7e8079d51d644212902ea5a305289571 P 6183387a03104ef6abd3836001c0dcdd: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4f3869b285954006b6ddf897a37b918f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } } }
I20260809 12:16:35.742286 23385 tablet_replica.cc:442] T 7e8079d51d644212902ea5a305289571 P 6183387a03104ef6abd3836001c0dcdd: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:16:35.743672 23377 tablet_replica.cc:439] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4f3869b285954006b6ddf897a37b918f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } } }
I20260809 12:16:35.743927 23377 tablet_replica.cc:442] T 7e8079d51d644212902ea5a305289571 P dfc9eb51ad9848f08a94c57a60c7c1c5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:16:35.753965 23380 tablet_replica.cc:439] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4f3869b285954006b6ddf897a37b918f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } } }
I20260809 12:16:35.754347 23380 tablet_replica.cc:442] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:16:35.753965 23423 tablet_replica.cc:439] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4f3869b285954006b6ddf897a37b918f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } } }
I20260809 12:16:35.755084 23423 tablet_replica.cc:442] T 7e8079d51d644212902ea5a305289571 P 4f3869b285954006b6ddf897a37b918f: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:16:35.774134 23409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:35.760258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37080 (local address 127.19.183.194:42423)
0809 12:16:35.760442 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:35.760449 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:35.767360 (+ 6911us) server_negotiation.cc:407] Connection header received
0809 12:16:35.767607 (+ 247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:35.767628 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:35.767746 (+ 118us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:35.767895 (+ 149us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:16:35.768408 (+ 513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.769464 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.771524 (+ 2060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.772142 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.772260 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:35.772783 (+ 523us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:16:35.773482 (+ 699us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:16:35.773626 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:35.773777 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 12:16:35.773863 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 12:16:35.791932 23442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:35.785838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:45405 (local address 127.19.183.194:57069)
0809 12:16:35.786545 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:35.786561 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:35.786682 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:35.787074 (+ 392us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:35.787077 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:35.787088 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:16:35.787326 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:35.787334 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.788780 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.788784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:35.789834 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:35.789845 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.790863 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.790867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:35.790973 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:35.791047 (+ 74us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:16:35.791479 (+ 432us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:16:35.791487 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 12:16:35.791546 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 12:16:35.791665 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":363,"threads_started":1}
I20260809 12:16:35.792013 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:35.785875 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:57069 (local address 127.19.183.195:45405)
0809 12:16:35.786061 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:35.786068 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:35.786657 (+ 589us) server_negotiation.cc:407] Connection header received
0809 12:16:35.786849 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:35.786855 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:35.786948 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:35.787050 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:16:35.787468 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.788634 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.789970 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.790730 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.790869 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:35.791251 (+ 382us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:16:35.791376 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:16:35.791473 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:35.791619 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 12:16:35.791705 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 12:16:35.835639 23409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:35.828283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:44257 (local address 127.19.183.194:42423)
0809 12:16:35.828477 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:35.828484 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:35.829254 (+ 770us) server_negotiation.cc:407] Connection header received
0809 12:16:35.829440 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:35.829446 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:35.829535 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:35.829637 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:16:35.830336 (+ 699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.831508 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.833778 (+ 2270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.834343 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.834442 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:35.835038 (+ 596us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:16:35.835139 (+ 101us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:16:35.835207 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:35.835370 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 12:16:35.835430 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 12:16:35.835906 23450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:35.828290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:42423 (local address 127.19.183.195:44257)
0809 12:16:35.829124 (+ 834us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:35.829144 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:35.829295 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:35.829681 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:35.829686 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:35.829699 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:16:35.830125 (+ 426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:35.830139 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.831704 (+ 1565us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.831714 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:35.833577 (+ 1863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:35.833593 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:35.834529 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:35.834536 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:35.834771 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:35.834851 (+ 80us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:16:35.835241 (+ 390us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:16:35.835254 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 12:16:35.835343 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 12:16:35.835480 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":640,"thread_start_us":447,"threads_started":1}
I20260809 12:16:35.882052 22682 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:16:35.882352 22682 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:16:35.886754 22682 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318: Sending DeleteTablet for 3 replicas of tablet e1feedd4cdda4a56b6fe59a4da0948ab
I20260809 12:16:35.887986 22880 tablet_service.cc:1514] Processing DeleteTablet for tablet e1feedd4cdda4a56b6fe59a4da0948ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:16:35 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:42514
I20260809 12:16:35.888914 23456 tablet_replica.cc:330] T e1feedd4cdda4a56b6fe59a4da0948ab P dfc9eb51ad9848f08a94c57a60c7c1c5: stopping tablet replica
I20260809 12:16:35.888983 23252 tablet_service.cc:1514] Processing DeleteTablet for tablet e1feedd4cdda4a56b6fe59a4da0948ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:16:35 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:38242
I20260809 12:16:35.889430 23456 raft_consensus.cc:2229] T e1feedd4cdda4a56b6fe59a4da0948ab P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:16:35.889886 23456 raft_consensus.cc:2258] T e1feedd4cdda4a56b6fe59a4da0948ab P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:16:35.890218 23061 tablet_service.cc:1514] Processing DeleteTablet for tablet e1feedd4cdda4a56b6fe59a4da0948ab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:16:35 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:37078
I20260809 12:16:35.891855 23456 ts_tablet_manager.cc:1905] T e1feedd4cdda4a56b6fe59a4da0948ab P dfc9eb51ad9848f08a94c57a60c7c1c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:16:35.897204 22682 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56652:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:16:35.897955 22682 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:16:35.899547 23456 ts_tablet_manager.cc:1918] T e1feedd4cdda4a56b6fe59a4da0948ab P dfc9eb51ad9848f08a94c57a60c7c1c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:16:35.899845 23456 log.cc:1196] T e1feedd4cdda4a56b6fe59a4da0948ab P dfc9eb51ad9848f08a94c57a60c7c1c5: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/e1feedd4cdda4a56b6fe59a4da0948ab
I20260809 12:16:35.903703 23456 ts_tablet_manager.cc:1939] T e1feedd4cdda4a56b6fe59a4da0948ab P dfc9eb51ad9848f08a94c57a60c7c1c5: Deleting consensus metadata
I20260809 12:16:35.906560 23250 tablet_service.cc:1467] Processing CreateTablet for tablet c072bff349794fcaa6ae53cac62307e5 (DEFAULT_TABLE table=test-table [id=befe964bd4b9479fb3071b160a75e4a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:35.907107 23250 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c072bff349794fcaa6ae53cac62307e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:35.907199 22880 tablet_service.cc:1467] Processing CreateTablet for tablet c072bff349794fcaa6ae53cac62307e5 (DEFAULT_TABLE table=test-table [id=befe964bd4b9479fb3071b160a75e4a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:35.907383 23060 tablet_service.cc:1467] Processing CreateTablet for tablet c072bff349794fcaa6ae53cac62307e5 (DEFAULT_TABLE table=test-table [id=befe964bd4b9479fb3071b160a75e4a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:35.907624 22880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c072bff349794fcaa6ae53cac62307e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:35.907603 22669 catalog_manager.cc:4841] TS dfc9eb51ad9848f08a94c57a60c7c1c5 (127.19.183.193:37075): tablet e1feedd4cdda4a56b6fe59a4da0948ab (table test-table [id=12e2b142eca349809724efa5b33e11e4]) successfully deleted
I20260809 12:16:35.907889 23060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c072bff349794fcaa6ae53cac62307e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:35.918807 23463 tablet_bootstrap.cc:492] T c072bff349794fcaa6ae53cac62307e5 P 4f3869b285954006b6ddf897a37b918f: Bootstrap starting.
I20260809 12:16:35.922456 23463 tablet_bootstrap.cc:654] T c072bff349794fcaa6ae53cac62307e5 P 4f3869b285954006b6ddf897a37b918f: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:35.925383 23462 tablet_bootstrap.cc:492] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5: Bootstrap starting.
I20260809 12:16:35.926884 23461 tablet_replica.cc:330] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd: stopping tablet replica
I20260809 12:16:35.928262 23462 tablet_bootstrap.cc:654] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:35.928766 23461 raft_consensus.cc:2229] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:16:35.929216 23460 tablet_replica.cc:330] T e1feedd4cdda4a56b6fe59a4da0948ab P 4f3869b285954006b6ddf897a37b918f: stopping tablet replica
I20260809 12:16:35.929656 23460 raft_consensus.cc:2229] T e1feedd4cdda4a56b6fe59a4da0948ab P 4f3869b285954006b6ddf897a37b918f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:16:35.929339 23465 tablet_bootstrap.cc:492] T c072bff349794fcaa6ae53cac62307e5 P 6183387a03104ef6abd3836001c0dcdd: Bootstrap starting.
I20260809 12:16:35.931537 23461 raft_consensus.cc:2258] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:16:35.933624 23465 tablet_bootstrap.cc:654] T c072bff349794fcaa6ae53cac62307e5 P 6183387a03104ef6abd3836001c0dcdd: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:35.934540 23462 tablet_bootstrap.cc:492] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5: No bootstrap required, opened a new log
I20260809 12:16:35.934836 23462 ts_tablet_manager.cc:1397] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260809 12:16:35.935879 23462 raft_consensus.cc:348] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:35.938251 23462 raft_consensus.cc:374] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:35.938442 23462 raft_consensus.cc:729] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfc9eb51ad9848f08a94c57a60c7c1c5, State: Initialized, Role: FOLLOWER
I20260809 12:16:35.939023 23460 raft_consensus.cc:2258] T e1feedd4cdda4a56b6fe59a4da0948ab P 4f3869b285954006b6ddf897a37b918f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:16:35.938905 23462 consensus_queue.cc:260] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:35.942070 23460 ts_tablet_manager.cc:1905] T e1feedd4cdda4a56b6fe59a4da0948ab P 4f3869b285954006b6ddf897a37b918f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:16:35.943692 23462 ts_tablet_manager.cc:1428] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5: Time spent starting tablet: real 0.009s user 0.002s sys 0.000s
I20260809 12:16:35.947592 23463 tablet_bootstrap.cc:492] T c072bff349794fcaa6ae53cac62307e5 P 4f3869b285954006b6ddf897a37b918f: No bootstrap required, opened a new log
I20260809 12:16:35.947845 23463 ts_tablet_manager.cc:1397] T c072bff349794fcaa6ae53cac62307e5 P 4f3869b285954006b6ddf897a37b918f: Time spent bootstrapping tablet: real 0.029s user 0.009s sys 0.003s
I20260809 12:16:35.938078 23461 ts_tablet_manager.cc:1905] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:16:35.949667 23465 tablet_bootstrap.cc:492] T c072bff349794fcaa6ae53cac62307e5 P 6183387a03104ef6abd3836001c0dcdd: No bootstrap required, opened a new log
I20260809 12:16:35.949882 23465 ts_tablet_manager.cc:1397] T c072bff349794fcaa6ae53cac62307e5 P 6183387a03104ef6abd3836001c0dcdd: Time spent bootstrapping tablet: real 0.021s user 0.003s sys 0.004s
I20260809 12:16:35.950860 23465 raft_consensus.cc:348] T c072bff349794fcaa6ae53cac62307e5 P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:35.951227 23465 raft_consensus.cc:374] T c072bff349794fcaa6ae53cac62307e5 P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:35.951393 23465 raft_consensus.cc:729] T c072bff349794fcaa6ae53cac62307e5 P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6183387a03104ef6abd3836001c0dcdd, State: Initialized, Role: FOLLOWER
I20260809 12:16:35.951758 23465 consensus_queue.cc:260] T c072bff349794fcaa6ae53cac62307e5 P 6183387a03104ef6abd3836001c0dcdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:35.954757 23465 ts_tablet_manager.cc:1428] T c072bff349794fcaa6ae53cac62307e5 P 6183387a03104ef6abd3836001c0dcdd: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260809 12:16:35.955181 23463 raft_consensus.cc:348] T c072bff349794fcaa6ae53cac62307e5 P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:35.956986 23463 raft_consensus.cc:374] T c072bff349794fcaa6ae53cac62307e5 P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:35.957130 23463 raft_consensus.cc:729] T c072bff349794fcaa6ae53cac62307e5 P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f3869b285954006b6ddf897a37b918f, State: Initialized, Role: FOLLOWER
I20260809 12:16:35.957412 23463 consensus_queue.cc:260] T c072bff349794fcaa6ae53cac62307e5 P 4f3869b285954006b6ddf897a37b918f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:35.958089 23463 ts_tablet_manager.cc:1428] T c072bff349794fcaa6ae53cac62307e5 P 4f3869b285954006b6ddf897a37b918f: Time spent starting tablet: real 0.010s user 0.000s sys 0.002s
I20260809 12:16:35.961741 23460 ts_tablet_manager.cc:1918] T e1feedd4cdda4a56b6fe59a4da0948ab P 4f3869b285954006b6ddf897a37b918f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:16:35.961916 23460 log.cc:1196] T e1feedd4cdda4a56b6fe59a4da0948ab P 4f3869b285954006b6ddf897a37b918f: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/e1feedd4cdda4a56b6fe59a4da0948ab
I20260809 12:16:35.962515 23460 ts_tablet_manager.cc:1939] T e1feedd4cdda4a56b6fe59a4da0948ab P 4f3869b285954006b6ddf897a37b918f: Deleting consensus metadata
I20260809 12:16:35.964397 22664 catalog_manager.cc:4841] TS 4f3869b285954006b6ddf897a37b918f (127.19.183.195:45405): tablet e1feedd4cdda4a56b6fe59a4da0948ab (table test-table [id=12e2b142eca349809724efa5b33e11e4]) successfully deleted
I20260809 12:16:35.965605 23461 ts_tablet_manager.cc:1918] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:16:35.965797 23461 log.cc:1196] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/e1feedd4cdda4a56b6fe59a4da0948ab
I20260809 12:16:35.966280 23461 ts_tablet_manager.cc:1939] T e1feedd4cdda4a56b6fe59a4da0948ab P 6183387a03104ef6abd3836001c0dcdd: Deleting consensus metadata
I20260809 12:16:35.967715 22669 catalog_manager.cc:4841] TS 6183387a03104ef6abd3836001c0dcdd (127.19.183.194:42423): tablet e1feedd4cdda4a56b6fe59a4da0948ab (table test-table [id=12e2b142eca349809724efa5b33e11e4]) successfully deleted
I20260809 12:16:36.119714 23377 raft_consensus.cc:482] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:16:36.119918 23377 raft_consensus.cc:504] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:36.120712 23377 leader_election.cc:288] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6183387a03104ef6abd3836001c0dcdd (127.19.183.194:42423), 4f3869b285954006b6ddf897a37b918f (127.19.183.195:45405)
I20260809 12:16:36.121296 23278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c072bff349794fcaa6ae53cac62307e5" candidate_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f3869b285954006b6ddf897a37b918f" is_pre_election: true
I20260809 12:16:36.121317 23085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c072bff349794fcaa6ae53cac62307e5" candidate_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6183387a03104ef6abd3836001c0dcdd" is_pre_election: true
I20260809 12:16:36.121564 23278 raft_consensus.cc:2454] T c072bff349794fcaa6ae53cac62307e5 P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfc9eb51ad9848f08a94c57a60c7c1c5 in term 0.
I20260809 12:16:36.121622 23085 raft_consensus.cc:2454] T c072bff349794fcaa6ae53cac62307e5 P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfc9eb51ad9848f08a94c57a60c7c1c5 in term 0.
I20260809 12:16:36.122123 22811 leader_election.cc:302] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f3869b285954006b6ddf897a37b918f, dfc9eb51ad9848f08a94c57a60c7c1c5; no voters:
I20260809 12:16:36.122462 23377 raft_consensus.cc:2781] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:16:36.122567 23377 raft_consensus.cc:482] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:16:36.122678 23377 raft_consensus.cc:3037] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:36.125815 23377 raft_consensus.cc:504] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:36.126523 23377 leader_election.cc:288] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5 [CANDIDATE]: Term 1 election: Requested vote from peers 6183387a03104ef6abd3836001c0dcdd (127.19.183.194:42423), 4f3869b285954006b6ddf897a37b918f (127.19.183.195:45405)
I20260809 12:16:36.127005 23278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c072bff349794fcaa6ae53cac62307e5" candidate_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f3869b285954006b6ddf897a37b918f"
I20260809 12:16:36.127023 23085 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c072bff349794fcaa6ae53cac62307e5" candidate_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6183387a03104ef6abd3836001c0dcdd"
I20260809 12:16:36.127249 23085 raft_consensus.cc:3037] T c072bff349794fcaa6ae53cac62307e5 P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:36.127249 23278 raft_consensus.cc:3037] T c072bff349794fcaa6ae53cac62307e5 P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:36.129633 23278 raft_consensus.cc:2454] T c072bff349794fcaa6ae53cac62307e5 P 4f3869b285954006b6ddf897a37b918f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfc9eb51ad9848f08a94c57a60c7c1c5 in term 1.
I20260809 12:16:36.130156 22811 leader_election.cc:302] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f3869b285954006b6ddf897a37b918f, dfc9eb51ad9848f08a94c57a60c7c1c5; no voters:
I20260809 12:16:36.130443 23377 raft_consensus.cc:2781] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:16:36.130475 23085 raft_consensus.cc:2454] T c072bff349794fcaa6ae53cac62307e5 P 6183387a03104ef6abd3836001c0dcdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfc9eb51ad9848f08a94c57a60c7c1c5 in term 1.
I20260809 12:16:36.132452 23377 raft_consensus.cc:686] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 1 LEADER]: Becoming Leader. State: Replica: dfc9eb51ad9848f08a94c57a60c7c1c5, State: Running, Role: LEADER
I20260809 12:16:36.133073 23377 consensus_queue.cc:237] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:36.136636 22682 catalog_manager.cc:5495] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to dfc9eb51ad9848f08a94c57a60c7c1c5 (127.19.183.193). New cstate: current_term: 1 leader_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } health_report { overall_health: UNKNOWN } } }
I20260809 12:16:36.171490 23471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:36.160326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42526 (local address 127.19.183.193:37075)
0809 12:16:36.160885 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:36.160889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:36.160903 (+ 14us) server_negotiation.cc:407] Connection header received
0809 12:16:36.160976 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:36.160979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:36.161059 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:36.161248 (+ 189us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:16:36.162130 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:36.163755 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:36.167546 (+ 3791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:36.168654 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:36.168822 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:36.169665 (+ 843us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:16:36.170432 (+ 767us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:16:36.170576 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:36.170878 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 12:16:36.170980 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":300,"threads_started":1}
I20260809 12:16:36.208981 23367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:36.192197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:59737 (local address 127.19.183.195:45405)
0809 12:16:36.192412 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:36.192420 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:36.194847 (+ 2427us) server_negotiation.cc:407] Connection header received
0809 12:16:36.195233 (+ 386us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:36.195241 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:36.195366 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:36.195437 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:16:36.196862 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:36.202523 (+ 5661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:36.206147 (+ 3624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:36.207171 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:36.207281 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:36.208131 (+ 850us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:16:36.208248 (+ 117us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:16:36.208324 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:36.208545 (+ 221us) server_negotiation.cc:299] Negotiation successful
0809 12:16:36.208613 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 12:16:36.209865 23475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:36.192661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:45405 (local address 127.19.183.193:59737)
0809 12:16:36.194660 (+ 1999us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:36.194684 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:36.195048 (+ 364us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:36.195996 (+ 948us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:36.196001 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:36.196016 (+ 15us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:16:36.196441 (+ 425us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:36.196458 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:36.202747 (+ 6289us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:36.202763 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:36.205952 (+ 3189us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:36.205986 (+ 34us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:36.207391 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:36.207400 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:36.207792 (+ 392us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:36.207885 (+ 93us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:16:36.208368 (+ 483us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:16:36.208381 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 12:16:36.208523 (+ 142us) client_negotiation.cc:240] Negotiation successful
0809 12:16:36.208702 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1743,"thread_start_us":1217,"threads_started":1}
I20260809 12:16:36.221617 23278 raft_consensus.cc:1261] T c072bff349794fcaa6ae53cac62307e5 P 4f3869b285954006b6ddf897a37b918f [term 1 FOLLOWER]: Refusing update from remote peer dfc9eb51ad9848f08a94c57a60c7c1c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:16:36.222649 23469 consensus_queue.cc:1035] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 12:16:36.226366 23087 raft_consensus.cc:1261] T c072bff349794fcaa6ae53cac62307e5 P 6183387a03104ef6abd3836001c0dcdd [term 1 FOLLOWER]: Refusing update from remote peer dfc9eb51ad9848f08a94c57a60c7c1c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:16:36.228084 23469 consensus_queue.cc:1035] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:16:36.254102 23417 mvcc.cc:205] Tried to move back new op lower bound from 7316593853316292608 to 7316593852964462592. Current Snapshot: MvccSnapshot[applied={T|T < 7316593853316292608}]
I20260809 12:16:36.289255 23471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:36.281078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:57991 (local address 127.19.183.193:37075)
0809 12:16:36.281264 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:36.281268 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:36.281284 (+ 16us) server_negotiation.cc:407] Connection header received
0809 12:16:36.281335 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:36.281339 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:36.281409 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:36.281523 (+ 114us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:16:36.282123 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:36.284241 (+ 2118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:36.286931 (+ 2690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:36.287664 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:36.287813 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:36.288348 (+ 535us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:16:36.288451 (+ 103us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:16:36.288562 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:36.288681 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 12:16:36.288760 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 12:16:36.289333 23450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:36.280418 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:37075 (local address 127.19.183.195:57991)
0809 12:16:36.280970 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:36.280992 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:36.281150 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:36.281537 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:36.281542 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:36.281554 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:16:36.281916 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:36.281927 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:36.284427 (+ 2500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:36.284433 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:36.286688 (+ 2255us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:36.286707 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:36.287824 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:36.287831 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:36.288092 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:36.288135 (+ 43us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:16:36.288566 (+ 431us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:16:36.288581 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 12:16:36.288662 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 12:16:36.288802 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321}
W20260809 12:16:36.290959 23417 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 12:16:37.923382 22681 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:16:37.923607 22681 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:16:37.929616 22681 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318: Sending DeleteTablet for 3 replicas of tablet c072bff349794fcaa6ae53cac62307e5
I20260809 12:16:37.930758 22880 tablet_service.cc:1514] Processing DeleteTablet for tablet c072bff349794fcaa6ae53cac62307e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:16:37 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:42514
I20260809 12:16:37.930820 23252 tablet_service.cc:1514] Processing DeleteTablet for tablet c072bff349794fcaa6ae53cac62307e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:16:37 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:38242
I20260809 12:16:37.930855 23060 tablet_service.cc:1514] Processing DeleteTablet for tablet c072bff349794fcaa6ae53cac62307e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:16:37 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:37078
I20260809 12:16:37.943513 23518 tablet_replica.cc:330] T c072bff349794fcaa6ae53cac62307e5 P 4f3869b285954006b6ddf897a37b918f: stopping tablet replica
I20260809 12:16:37.944135 23517 tablet_replica.cc:330] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5: stopping tablet replica
I20260809 12:16:37.944064 23518 raft_consensus.cc:2229] T c072bff349794fcaa6ae53cac62307e5 P 4f3869b285954006b6ddf897a37b918f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:16:37.944378 23518 raft_consensus.cc:2258] T c072bff349794fcaa6ae53cac62307e5 P 4f3869b285954006b6ddf897a37b918f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:16:37.944486 23517 raft_consensus.cc:2229] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:16:37.945246 23517 raft_consensus.cc:2258] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:16:37.945653 23518 ts_tablet_manager.cc:1905] T c072bff349794fcaa6ae53cac62307e5 P 4f3869b285954006b6ddf897a37b918f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:16:37.946259 23519 tablet_replica.cc:330] T c072bff349794fcaa6ae53cac62307e5 P 6183387a03104ef6abd3836001c0dcdd: stopping tablet replica
I20260809 12:16:37.946573 23519 raft_consensus.cc:2229] T c072bff349794fcaa6ae53cac62307e5 P 6183387a03104ef6abd3836001c0dcdd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:16:37.946952 23519 raft_consensus.cc:2258] T c072bff349794fcaa6ae53cac62307e5 P 6183387a03104ef6abd3836001c0dcdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:16:37.947890 23517 ts_tablet_manager.cc:1905] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:16:37.952715 23519 ts_tablet_manager.cc:1905] T c072bff349794fcaa6ae53cac62307e5 P 6183387a03104ef6abd3836001c0dcdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:16:37.955308 23518 ts_tablet_manager.cc:1918] T c072bff349794fcaa6ae53cac62307e5 P 4f3869b285954006b6ddf897a37b918f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 12:16:37.955616 23518 log.cc:1196] T c072bff349794fcaa6ae53cac62307e5 P 4f3869b285954006b6ddf897a37b918f: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/c072bff349794fcaa6ae53cac62307e5
I20260809 12:16:37.956291 23518 ts_tablet_manager.cc:1939] T c072bff349794fcaa6ae53cac62307e5 P 4f3869b285954006b6ddf897a37b918f: Deleting consensus metadata
I20260809 12:16:37.957628 22664 catalog_manager.cc:4841] TS 4f3869b285954006b6ddf897a37b918f (127.19.183.195:45405): tablet c072bff349794fcaa6ae53cac62307e5 (table test-table [id=befe964bd4b9479fb3071b160a75e4a0]) successfully deleted
I20260809 12:16:37.960861 23519 ts_tablet_manager.cc:1918] T c072bff349794fcaa6ae53cac62307e5 P 6183387a03104ef6abd3836001c0dcdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 12:16:37.961088 23519 log.cc:1196] T c072bff349794fcaa6ae53cac62307e5 P 6183387a03104ef6abd3836001c0dcdd: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/c072bff349794fcaa6ae53cac62307e5
I20260809 12:16:37.961104 23517 ts_tablet_manager.cc:1918] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 12:16:37.961364 23517 log.cc:1196] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/c072bff349794fcaa6ae53cac62307e5
I20260809 12:16:37.961800 23519 ts_tablet_manager.cc:1939] T c072bff349794fcaa6ae53cac62307e5 P 6183387a03104ef6abd3836001c0dcdd: Deleting consensus metadata
I20260809 12:16:37.961926 23517 ts_tablet_manager.cc:1939] T c072bff349794fcaa6ae53cac62307e5 P dfc9eb51ad9848f08a94c57a60c7c1c5: Deleting consensus metadata
I20260809 12:16:37.962987 22669 catalog_manager.cc:4841] TS dfc9eb51ad9848f08a94c57a60c7c1c5 (127.19.183.193:37075): tablet c072bff349794fcaa6ae53cac62307e5 (table test-table [id=befe964bd4b9479fb3071b160a75e4a0]) successfully deleted
I20260809 12:16:37.963274 22669 catalog_manager.cc:4841] TS 6183387a03104ef6abd3836001c0dcdd (127.19.183.194:42423): tablet c072bff349794fcaa6ae53cac62307e5 (table test-table [id=befe964bd4b9479fb3071b160a75e4a0]) successfully deleted
Aug 09 12:16:37 dist-test-slave-39l8 krb5kdc[22631](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277797, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 12:16:37 dist-test-slave-39l8 krb5kdc[22631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277797, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:38.002218 23530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:37.986983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56668 (local address 127.19.183.254:45709)
0809 12:16:37.987539 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:37.987543 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:37.987566 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:16:37.987736 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:37.987740 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:37.987805 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:37.987919 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:37.989304 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:37.990034 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:37.991445 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:37.991841 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:37.991942 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:37.995807 (+ 3865us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:37.995812 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:37.995815 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:37.995850 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:37.999465 (+ 3615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:38.000093 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:38.000097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:38.000099 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:38.000287 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:38.000607 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:38.000610 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:38.000612 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:38.000962 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:38.001249 (+ 287us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:38.001460 (+ 211us) server_negotiation.cc:299] Negotiation successful
0809 12:16:38.001700 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":436,"thread_start_us":282,"threads_started":1}
I20260809 12:16:38.006034 22681 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56668:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:16:38.006913 22681 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:16:38.020752 23252 tablet_service.cc:1467] Processing CreateTablet for tablet 6215ed9fe0004bfaa90aa3ac4c820fff (DEFAULT_TABLE table=test-table [id=00fd0f8f84624aa0ac32ffa223442672]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:38.020918 23060 tablet_service.cc:1467] Processing CreateTablet for tablet 6215ed9fe0004bfaa90aa3ac4c820fff (DEFAULT_TABLE table=test-table [id=00fd0f8f84624aa0ac32ffa223442672]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:38.021418 23252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6215ed9fe0004bfaa90aa3ac4c820fff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:38.021580 22880 tablet_service.cc:1467] Processing CreateTablet for tablet 6215ed9fe0004bfaa90aa3ac4c820fff (DEFAULT_TABLE table=test-table [id=00fd0f8f84624aa0ac32ffa223442672]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:38.022091 22880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6215ed9fe0004bfaa90aa3ac4c820fff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:38.022904 23060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6215ed9fe0004bfaa90aa3ac4c820fff. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:38.031495 23531 tablet_bootstrap.cc:492] T 6215ed9fe0004bfaa90aa3ac4c820fff P 4f3869b285954006b6ddf897a37b918f: Bootstrap starting.
I20260809 12:16:38.036298 23531 tablet_bootstrap.cc:654] T 6215ed9fe0004bfaa90aa3ac4c820fff P 4f3869b285954006b6ddf897a37b918f: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:38.039239 23532 tablet_bootstrap.cc:492] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5: Bootstrap starting.
I20260809 12:16:38.044149 23532 tablet_bootstrap.cc:654] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:38.048401 23533 tablet_bootstrap.cc:492] T 6215ed9fe0004bfaa90aa3ac4c820fff P 6183387a03104ef6abd3836001c0dcdd: Bootstrap starting.
I20260809 12:16:38.050158 23531 tablet_bootstrap.cc:492] T 6215ed9fe0004bfaa90aa3ac4c820fff P 4f3869b285954006b6ddf897a37b918f: No bootstrap required, opened a new log
I20260809 12:16:38.050408 23531 ts_tablet_manager.cc:1397] T 6215ed9fe0004bfaa90aa3ac4c820fff P 4f3869b285954006b6ddf897a37b918f: Time spent bootstrapping tablet: real 0.019s user 0.004s sys 0.010s
I20260809 12:16:38.051789 23532 tablet_bootstrap.cc:492] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5: No bootstrap required, opened a new log
I20260809 12:16:38.052085 23533 tablet_bootstrap.cc:654] T 6215ed9fe0004bfaa90aa3ac4c820fff P 6183387a03104ef6abd3836001c0dcdd: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:38.051863 23531 raft_consensus.cc:348] T 6215ed9fe0004bfaa90aa3ac4c820fff P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:38.052363 23531 raft_consensus.cc:374] T 6215ed9fe0004bfaa90aa3ac4c820fff P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:38.052418 23532 ts_tablet_manager.cc:1397] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5: Time spent bootstrapping tablet: real 0.013s user 0.005s sys 0.006s
I20260809 12:16:38.052650 23531 raft_consensus.cc:729] T 6215ed9fe0004bfaa90aa3ac4c820fff P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f3869b285954006b6ddf897a37b918f, State: Initialized, Role: FOLLOWER
I20260809 12:16:38.053121 23531 consensus_queue.cc:260] T 6215ed9fe0004bfaa90aa3ac4c820fff P 4f3869b285954006b6ddf897a37b918f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:38.054778 23532 raft_consensus.cc:348] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:38.055251 23532 raft_consensus.cc:374] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:38.055524 23532 raft_consensus.cc:729] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfc9eb51ad9848f08a94c57a60c7c1c5, State: Initialized, Role: FOLLOWER
I20260809 12:16:38.055967 23532 consensus_queue.cc:260] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:38.056861 23532 ts_tablet_manager.cc:1428] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260809 12:16:38.058883 23531 ts_tablet_manager.cc:1428] T 6215ed9fe0004bfaa90aa3ac4c820fff P 4f3869b285954006b6ddf897a37b918f: Time spent starting tablet: real 0.008s user 0.004s sys 0.000s
I20260809 12:16:38.062510 23533 tablet_bootstrap.cc:492] T 6215ed9fe0004bfaa90aa3ac4c820fff P 6183387a03104ef6abd3836001c0dcdd: No bootstrap required, opened a new log
I20260809 12:16:38.062788 23533 ts_tablet_manager.cc:1397] T 6215ed9fe0004bfaa90aa3ac4c820fff P 6183387a03104ef6abd3836001c0dcdd: Time spent bootstrapping tablet: real 0.015s user 0.004s sys 0.008s
I20260809 12:16:38.064208 23533 raft_consensus.cc:348] T 6215ed9fe0004bfaa90aa3ac4c820fff P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:38.064670 23533 raft_consensus.cc:374] T 6215ed9fe0004bfaa90aa3ac4c820fff P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:38.064908 23533 raft_consensus.cc:729] T 6215ed9fe0004bfaa90aa3ac4c820fff P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6183387a03104ef6abd3836001c0dcdd, State: Initialized, Role: FOLLOWER
I20260809 12:16:38.065220 23533 consensus_queue.cc:260] T 6215ed9fe0004bfaa90aa3ac4c820fff P 6183387a03104ef6abd3836001c0dcdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:38.066777 23533 ts_tablet_manager.cc:1428] T 6215ed9fe0004bfaa90aa3ac4c820fff P 6183387a03104ef6abd3836001c0dcdd: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260809 12:16:38.157341 23507 raft_consensus.cc:482] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:16:38.157702 23507 raft_consensus.cc:504] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:38.158870 23507 leader_election.cc:288] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6183387a03104ef6abd3836001c0dcdd (127.19.183.194:42423), 4f3869b285954006b6ddf897a37b918f (127.19.183.195:45405)
I20260809 12:16:38.159561 23279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6215ed9fe0004bfaa90aa3ac4c820fff" candidate_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f3869b285954006b6ddf897a37b918f" is_pre_election: true
I20260809 12:16:38.159631 23087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6215ed9fe0004bfaa90aa3ac4c820fff" candidate_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6183387a03104ef6abd3836001c0dcdd" is_pre_election: true
I20260809 12:16:38.159983 23279 raft_consensus.cc:2454] T 6215ed9fe0004bfaa90aa3ac4c820fff P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfc9eb51ad9848f08a94c57a60c7c1c5 in term 0.
I20260809 12:16:38.160056 23087 raft_consensus.cc:2454] T 6215ed9fe0004bfaa90aa3ac4c820fff P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfc9eb51ad9848f08a94c57a60c7c1c5 in term 0.
I20260809 12:16:38.160674 22811 leader_election.cc:302] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f3869b285954006b6ddf897a37b918f, dfc9eb51ad9848f08a94c57a60c7c1c5; no voters:
I20260809 12:16:38.161123 23507 raft_consensus.cc:2781] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:16:38.161372 23507 raft_consensus.cc:482] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:16:38.161608 23507 raft_consensus.cc:3037] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:38.164880 23507 raft_consensus.cc:504] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:38.165567 23507 leader_election.cc:288] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5 [CANDIDATE]: Term 1 election: Requested vote from peers 6183387a03104ef6abd3836001c0dcdd (127.19.183.194:42423), 4f3869b285954006b6ddf897a37b918f (127.19.183.195:45405)
I20260809 12:16:38.166031 23087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6215ed9fe0004bfaa90aa3ac4c820fff" candidate_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6183387a03104ef6abd3836001c0dcdd"
I20260809 12:16:38.166118 23279 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6215ed9fe0004bfaa90aa3ac4c820fff" candidate_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f3869b285954006b6ddf897a37b918f"
I20260809 12:16:38.166419 23087 raft_consensus.cc:3037] T 6215ed9fe0004bfaa90aa3ac4c820fff P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:38.166431 23279 raft_consensus.cc:3037] T 6215ed9fe0004bfaa90aa3ac4c820fff P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:38.169178 23279 raft_consensus.cc:2454] T 6215ed9fe0004bfaa90aa3ac4c820fff P 4f3869b285954006b6ddf897a37b918f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfc9eb51ad9848f08a94c57a60c7c1c5 in term 1.
I20260809 12:16:38.169402 23087 raft_consensus.cc:2454] T 6215ed9fe0004bfaa90aa3ac4c820fff P 6183387a03104ef6abd3836001c0dcdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfc9eb51ad9848f08a94c57a60c7c1c5 in term 1.
I20260809 12:16:38.169721 22811 leader_election.cc:302] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f3869b285954006b6ddf897a37b918f, dfc9eb51ad9848f08a94c57a60c7c1c5; no voters:
I20260809 12:16:38.170094 23507 raft_consensus.cc:2781] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:16:38.170284 23507 raft_consensus.cc:686] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 1 LEADER]: Becoming Leader. State: Replica: dfc9eb51ad9848f08a94c57a60c7c1c5, State: Running, Role: LEADER
I20260809 12:16:38.170679 23507 consensus_queue.cc:237] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:38.174482 22682 catalog_manager.cc:5495] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to dfc9eb51ad9848f08a94c57a60c7c1c5 (127.19.183.193). New cstate: current_term: 1 leader_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } health_report { overall_health: UNKNOWN } } }
I20260809 12:16:38.236593 23540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:38.227559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42540 (local address 127.19.183.193:37075)
0809 12:16:38.229370 (+ 1811us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:38.229374 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:38.229409 (+ 35us) server_negotiation.cc:407] Connection header received
0809 12:16:38.229463 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:38.229466 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:38.229524 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:38.229610 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:16:38.230336 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:38.231527 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:38.233931 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:38.234642 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:38.234832 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:38.235537 (+ 705us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:16:38.235651 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:16:38.235721 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:38.235908 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 12:16:38.235972 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1656,"thread_start_us":1432,"threads_started":1}
I20260809 12:16:38.247205 23085 raft_consensus.cc:1261] T 6215ed9fe0004bfaa90aa3ac4c820fff P 6183387a03104ef6abd3836001c0dcdd [term 1 FOLLOWER]: Refusing update from remote peer dfc9eb51ad9848f08a94c57a60c7c1c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:16:38.247213 23279 raft_consensus.cc:1261] T 6215ed9fe0004bfaa90aa3ac4c820fff P 4f3869b285954006b6ddf897a37b918f [term 1 FOLLOWER]: Refusing update from remote peer dfc9eb51ad9848f08a94c57a60c7c1c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:16:38.247946 23507 consensus_queue.cc:1035] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:16:38.248427 23486 consensus_queue.cc:1035] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 12:16:38.303861 22683 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:16:38.304199 22683 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:16:38.307600 22683 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318: Sending DeleteTablet for 3 replicas of tablet 6215ed9fe0004bfaa90aa3ac4c820fff
I20260809 12:16:38.308539 22880 tablet_service.cc:1514] Processing DeleteTablet for tablet 6215ed9fe0004bfaa90aa3ac4c820fff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:16:38 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:42514
I20260809 12:16:38.308671 23060 tablet_service.cc:1514] Processing DeleteTablet for tablet 6215ed9fe0004bfaa90aa3ac4c820fff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:16:38 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:37078
I20260809 12:16:38.308780 23252 tablet_service.cc:1514] Processing DeleteTablet for tablet 6215ed9fe0004bfaa90aa3ac4c820fff with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:16:38 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:38242
I20260809 12:16:38.309041 23519 tablet_replica.cc:330] T 6215ed9fe0004bfaa90aa3ac4c820fff P 6183387a03104ef6abd3836001c0dcdd: stopping tablet replica
I20260809 12:16:38.309038 23517 tablet_replica.cc:330] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5: stopping tablet replica
I20260809 12:16:38.309209 23518 tablet_replica.cc:330] T 6215ed9fe0004bfaa90aa3ac4c820fff P 4f3869b285954006b6ddf897a37b918f: stopping tablet replica
I20260809 12:16:38.309374 23519 raft_consensus.cc:2229] T 6215ed9fe0004bfaa90aa3ac4c820fff P 6183387a03104ef6abd3836001c0dcdd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:16:38.309367 23517 raft_consensus.cc:2229] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:16:38.309506 23518 raft_consensus.cc:2229] T 6215ed9fe0004bfaa90aa3ac4c820fff P 4f3869b285954006b6ddf897a37b918f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:16:38.309638 23519 raft_consensus.cc:2258] T 6215ed9fe0004bfaa90aa3ac4c820fff P 6183387a03104ef6abd3836001c0dcdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:16:38.309769 23517 raft_consensus.cc:2258] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:16:38.309773 23518 raft_consensus.cc:2258] T 6215ed9fe0004bfaa90aa3ac4c820fff P 4f3869b285954006b6ddf897a37b918f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:16:38.310865 23519 ts_tablet_manager.cc:1905] T 6215ed9fe0004bfaa90aa3ac4c820fff P 6183387a03104ef6abd3836001c0dcdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:16:38.310869 23517 ts_tablet_manager.cc:1905] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:16:38.310626 22683 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56668:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 12:16:38.311192 23518 ts_tablet_manager.cc:1905] T 6215ed9fe0004bfaa90aa3ac4c820fff P 4f3869b285954006b6ddf897a37b918f: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 12:16:38.311712 22683 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:16:38.318667 23517 ts_tablet_manager.cc:1918] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:16:38.319267 23517 log.cc:1196] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/6215ed9fe0004bfaa90aa3ac4c820fff
I20260809 12:16:38.319835 23517 ts_tablet_manager.cc:1939] T 6215ed9fe0004bfaa90aa3ac4c820fff P dfc9eb51ad9848f08a94c57a60c7c1c5: Deleting consensus metadata
I20260809 12:16:38.320281 23252 tablet_service.cc:1467] Processing CreateTablet for tablet 4bcd9e0b63dd40a0a8780e9e187f7175 (DEFAULT_TABLE table=test-table [id=7fc60e49a92a444f9f7ab30e02992b15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:38.320492 22880 tablet_service.cc:1467] Processing CreateTablet for tablet 4bcd9e0b63dd40a0a8780e9e187f7175 (DEFAULT_TABLE table=test-table [id=7fc60e49a92a444f9f7ab30e02992b15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:38.320777 23252 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bcd9e0b63dd40a0a8780e9e187f7175. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:38.321003 22880 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bcd9e0b63dd40a0a8780e9e187f7175. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:38.321142 23060 tablet_service.cc:1467] Processing CreateTablet for tablet 4bcd9e0b63dd40a0a8780e9e187f7175 (DEFAULT_TABLE table=test-table [id=7fc60e49a92a444f9f7ab30e02992b15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:38.321717 23060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4bcd9e0b63dd40a0a8780e9e187f7175. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:38.326028 23519 ts_tablet_manager.cc:1918] T 6215ed9fe0004bfaa90aa3ac4c820fff P 6183387a03104ef6abd3836001c0dcdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:16:38.326576 23518 ts_tablet_manager.cc:1918] T 6215ed9fe0004bfaa90aa3ac4c820fff P 4f3869b285954006b6ddf897a37b918f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:16:38.326653 23519 log.cc:1196] T 6215ed9fe0004bfaa90aa3ac4c820fff P 6183387a03104ef6abd3836001c0dcdd: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/6215ed9fe0004bfaa90aa3ac4c820fff
I20260809 12:16:38.327042 23518 log.cc:1196] T 6215ed9fe0004bfaa90aa3ac4c820fff P 4f3869b285954006b6ddf897a37b918f: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/6215ed9fe0004bfaa90aa3ac4c820fff
I20260809 12:16:38.327687 23518 ts_tablet_manager.cc:1939] T 6215ed9fe0004bfaa90aa3ac4c820fff P 4f3869b285954006b6ddf897a37b918f: Deleting consensus metadata
I20260809 12:16:38.329049 23532 tablet_bootstrap.cc:492] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5: Bootstrap starting.
I20260809 12:16:38.329574 22664 catalog_manager.cc:4841] TS 4f3869b285954006b6ddf897a37b918f (127.19.183.195:45405): tablet 6215ed9fe0004bfaa90aa3ac4c820fff (table test-table [id=00fd0f8f84624aa0ac32ffa223442672]) successfully deleted
I20260809 12:16:38.330272 23519 ts_tablet_manager.cc:1939] T 6215ed9fe0004bfaa90aa3ac4c820fff P 6183387a03104ef6abd3836001c0dcdd: Deleting consensus metadata
I20260809 12:16:38.330899 23533 tablet_bootstrap.cc:492] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 6183387a03104ef6abd3836001c0dcdd: Bootstrap starting.
I20260809 12:16:38.330955 23531 tablet_bootstrap.cc:492] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 4f3869b285954006b6ddf897a37b918f: Bootstrap starting.
I20260809 12:16:38.331593 22669 catalog_manager.cc:4841] TS 6183387a03104ef6abd3836001c0dcdd (127.19.183.194:42423): tablet 6215ed9fe0004bfaa90aa3ac4c820fff (table test-table [id=00fd0f8f84624aa0ac32ffa223442672]) successfully deleted
I20260809 12:16:38.332768 23532 tablet_bootstrap.cc:654] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:38.333194 23533 tablet_bootstrap.cc:654] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 6183387a03104ef6abd3836001c0dcdd: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:38.333370 23531 tablet_bootstrap.cc:654] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 4f3869b285954006b6ddf897a37b918f: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:38.336257 22669 catalog_manager.cc:4841] TS dfc9eb51ad9848f08a94c57a60c7c1c5 (127.19.183.193:37075): tablet 6215ed9fe0004bfaa90aa3ac4c820fff (table test-table [id=00fd0f8f84624aa0ac32ffa223442672]) successfully deleted
I20260809 12:16:38.337728 23532 tablet_bootstrap.cc:492] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5: No bootstrap required, opened a new log
I20260809 12:16:38.337741 23531 tablet_bootstrap.cc:492] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 4f3869b285954006b6ddf897a37b918f: No bootstrap required, opened a new log
I20260809 12:16:38.337930 23533 tablet_bootstrap.cc:492] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 6183387a03104ef6abd3836001c0dcdd: No bootstrap required, opened a new log
I20260809 12:16:38.337971 23532 ts_tablet_manager.cc:1397] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5: Time spent bootstrapping tablet: real 0.009s user 0.005s sys 0.000s
I20260809 12:16:38.337971 23531 ts_tablet_manager.cc:1397] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 4f3869b285954006b6ddf897a37b918f: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260809 12:16:38.338128 23533 ts_tablet_manager.cc:1397] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 6183387a03104ef6abd3836001c0dcdd: Time spent bootstrapping tablet: real 0.007s user 0.001s sys 0.004s
I20260809 12:16:38.339028 23533 raft_consensus.cc:348] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:38.339131 23532 raft_consensus.cc:348] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:38.339320 23533 raft_consensus.cc:374] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:38.339145 23531 raft_consensus.cc:348] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:38.339461 23533 raft_consensus.cc:729] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6183387a03104ef6abd3836001c0dcdd, State: Initialized, Role: FOLLOWER
I20260809 12:16:38.339483 23532 raft_consensus.cc:374] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:38.339596 23531 raft_consensus.cc:374] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:38.339675 23532 raft_consensus.cc:729] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfc9eb51ad9848f08a94c57a60c7c1c5, State: Initialized, Role: FOLLOWER
I20260809 12:16:38.339758 23531 raft_consensus.cc:729] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f3869b285954006b6ddf897a37b918f, State: Initialized, Role: FOLLOWER
I20260809 12:16:38.339896 23533 consensus_queue.cc:260] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 6183387a03104ef6abd3836001c0dcdd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:38.340049 23532 consensus_queue.cc:260] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:38.340086 23531 consensus_queue.cc:260] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 4f3869b285954006b6ddf897a37b918f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:38.340844 23532 ts_tablet_manager.cc:1428] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 12:16:38.341470 23531 ts_tablet_manager.cc:1428] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 4f3869b285954006b6ddf897a37b918f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 12:16:38.342216 23533 ts_tablet_manager.cc:1428] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 6183387a03104ef6abd3836001c0dcdd: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 12:16:38.493666 23548 raft_consensus.cc:482] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:16:38.493928 23548 raft_consensus.cc:504] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:38.494935 23548 leader_election.cc:288] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6183387a03104ef6abd3836001c0dcdd (127.19.183.194:42423), 4f3869b285954006b6ddf897a37b918f (127.19.183.195:45405)
I20260809 12:16:38.495551 23278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bcd9e0b63dd40a0a8780e9e187f7175" candidate_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f3869b285954006b6ddf897a37b918f" is_pre_election: true
I20260809 12:16:38.495620 23087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bcd9e0b63dd40a0a8780e9e187f7175" candidate_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6183387a03104ef6abd3836001c0dcdd" is_pre_election: true
I20260809 12:16:38.495950 23278 raft_consensus.cc:2454] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfc9eb51ad9848f08a94c57a60c7c1c5 in term 0.
I20260809 12:16:38.495954 23087 raft_consensus.cc:2454] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfc9eb51ad9848f08a94c57a60c7c1c5 in term 0.
I20260809 12:16:38.496518 22817 leader_election.cc:302] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6183387a03104ef6abd3836001c0dcdd, dfc9eb51ad9848f08a94c57a60c7c1c5; no voters:
I20260809 12:16:38.496898 23548 raft_consensus.cc:2781] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:16:38.497102 23548 raft_consensus.cc:482] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:16:38.497279 23548 raft_consensus.cc:3037] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:38.500082 23548 raft_consensus.cc:504] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:38.500696 23548 leader_election.cc:288] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5 [CANDIDATE]: Term 1 election: Requested vote from peers 6183387a03104ef6abd3836001c0dcdd (127.19.183.194:42423), 4f3869b285954006b6ddf897a37b918f (127.19.183.195:45405)
I20260809 12:16:38.501366 23087 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bcd9e0b63dd40a0a8780e9e187f7175" candidate_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6183387a03104ef6abd3836001c0dcdd"
I20260809 12:16:38.501386 23278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4bcd9e0b63dd40a0a8780e9e187f7175" candidate_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f3869b285954006b6ddf897a37b918f"
I20260809 12:16:38.501689 23087 raft_consensus.cc:3037] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 6183387a03104ef6abd3836001c0dcdd [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:38.501762 23278 raft_consensus.cc:3037] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 4f3869b285954006b6ddf897a37b918f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:38.504714 23278 raft_consensus.cc:2454] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 4f3869b285954006b6ddf897a37b918f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfc9eb51ad9848f08a94c57a60c7c1c5 in term 1.
I20260809 12:16:38.505065 23087 raft_consensus.cc:2454] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 6183387a03104ef6abd3836001c0dcdd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfc9eb51ad9848f08a94c57a60c7c1c5 in term 1.
I20260809 12:16:38.505388 22811 leader_election.cc:302] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f3869b285954006b6ddf897a37b918f, dfc9eb51ad9848f08a94c57a60c7c1c5; no voters:
I20260809 12:16:38.505795 23548 raft_consensus.cc:2781] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:16:38.505988 23548 raft_consensus.cc:686] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 1 LEADER]: Becoming Leader. State: Replica: dfc9eb51ad9848f08a94c57a60c7c1c5, State: Running, Role: LEADER
I20260809 12:16:38.506455 23548 consensus_queue.cc:237] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } }
I20260809 12:16:38.510185 22680 catalog_manager.cc:5495] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5 reported cstate change: term changed from 0 to 1, leader changed from <none> to dfc9eb51ad9848f08a94c57a60c7c1c5 (127.19.183.193). New cstate: current_term: 1 leader_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dfc9eb51ad9848f08a94c57a60c7c1c5" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37075 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 } health_report { overall_health: UNKNOWN } } }
I20260809 12:16:38.535707 23087 raft_consensus.cc:1261] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 6183387a03104ef6abd3836001c0dcdd [term 1 FOLLOWER]: Refusing update from remote peer dfc9eb51ad9848f08a94c57a60c7c1c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:16:38.535813 23278 raft_consensus.cc:1261] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 4f3869b285954006b6ddf897a37b918f [term 1 FOLLOWER]: Refusing update from remote peer dfc9eb51ad9848f08a94c57a60c7c1c5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:16:38.536369 23486 consensus_queue.cc:1035] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f3869b285954006b6ddf897a37b918f" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 45405 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:16:38.536765 23548 consensus_queue.cc:1035] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6183387a03104ef6abd3836001c0dcdd" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 12:16:38.559470 23494 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 12:16:40.145609 22683 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:16:40.145920 22683 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56668:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:16:40.151262 22683 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d975166788fd4936a3544e7d854a0318: Sending DeleteTablet for 3 replicas of tablet 4bcd9e0b63dd40a0a8780e9e187f7175
I20260809 12:16:40.152035 22880 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bcd9e0b63dd40a0a8780e9e187f7175 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:16:40 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:42514
I20260809 12:16:40.152230 23060 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bcd9e0b63dd40a0a8780e9e187f7175 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:16:40 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:37078
I20260809 12:16:40.152323 23252 tablet_service.cc:1514] Processing DeleteTablet for tablet 4bcd9e0b63dd40a0a8780e9e187f7175 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:16:40 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:38242
I20260809 12:16:40.153978 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 22782
I20260809 12:16:40.154527 23586 tablet_replica.cc:330] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 4f3869b285954006b6ddf897a37b918f: stopping tablet replica
I20260809 12:16:40.154846 23586 raft_consensus.cc:2229] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 4f3869b285954006b6ddf897a37b918f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:16:40.154929 23584 tablet_replica.cc:330] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 6183387a03104ef6abd3836001c0dcdd: stopping tablet replica
I20260809 12:16:40.155197 23586 raft_consensus.cc:2258] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 4f3869b285954006b6ddf897a37b918f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:16:40.155254 23584 raft_consensus.cc:2229] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 6183387a03104ef6abd3836001c0dcdd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:16:40.155601 23584 raft_consensus.cc:2258] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 6183387a03104ef6abd3836001c0dcdd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:16:40.156625 23586 ts_tablet_manager.cc:1905] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 4f3869b285954006b6ddf897a37b918f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:16:40.156632 23584 ts_tablet_manager.cc:1905] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 6183387a03104ef6abd3836001c0dcdd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:16:40.164443 23584 ts_tablet_manager.cc:1918] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 6183387a03104ef6abd3836001c0dcdd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 12:16:40.164667 23584 log.cc:1196] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 6183387a03104ef6abd3836001c0dcdd: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/4bcd9e0b63dd40a0a8780e9e187f7175
I20260809 12:16:40.164875 23586 ts_tablet_manager.cc:1918] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 4f3869b285954006b6ddf897a37b918f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 12:16:40.165091 23586 log.cc:1196] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 4f3869b285954006b6ddf897a37b918f: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/4bcd9e0b63dd40a0a8780e9e187f7175
I20260809 12:16:40.165289 23584 ts_tablet_manager.cc:1939] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 6183387a03104ef6abd3836001c0dcdd: Deleting consensus metadata
I20260809 12:16:40.165773 23586 ts_tablet_manager.cc:1939] T 4bcd9e0b63dd40a0a8780e9e187f7175 P 4f3869b285954006b6ddf897a37b918f: Deleting consensus metadata
I20260809 12:16:40.166472 23583 tablet_replica.cc:330] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5: stopping tablet replica
I20260809 12:16:40.166556 22669 catalog_manager.cc:4841] TS 6183387a03104ef6abd3836001c0dcdd (127.19.183.194:42423): tablet 4bcd9e0b63dd40a0a8780e9e187f7175 (table test-table [id=7fc60e49a92a444f9f7ab30e02992b15]) successfully deleted
I20260809 12:16:40.166884 23583 raft_consensus.cc:2229] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:16:40.167405 23583 raft_consensus.cc:2258] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:16:40.167429 22664 catalog_manager.cc:4841] TS 4f3869b285954006b6ddf897a37b918f (127.19.183.195:45405): tablet 4bcd9e0b63dd40a0a8780e9e187f7175 (table test-table [id=7fc60e49a92a444f9f7ab30e02992b15]) successfully deleted
I20260809 12:16:40.168519 23583 ts_tablet_manager.cc:1905] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:16:40.175210 23583 ts_tablet_manager.cc:1918] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 12:16:40.175340 23583 log.cc:1196] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/4bcd9e0b63dd40a0a8780e9e187f7175
I20260809 12:16:40.175820 23583 ts_tablet_manager.cc:1939] T 4bcd9e0b63dd40a0a8780e9e187f7175 P dfc9eb51ad9848f08a94c57a60c7c1c5: Deleting consensus metadata
I20260809 12:16:40.176631 22669 catalog_manager.cc:4841] TS dfc9eb51ad9848f08a94c57a60c7c1c5 (127.19.183.193:37075): tablet 4bcd9e0b63dd40a0a8780e9e187f7175 (table test-table [id=7fc60e49a92a444f9f7ab30e02992b15]) successfully deleted
Aug 09 12:16:40 dist-test-slave-39l8 krb5kdc[22631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277797, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.183.193@KRBTEST.COM
I20260809 12:16:40.189432 23589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:40.156020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42550 (local address 127.19.183.193:37075)
0809 12:16:40.166814 (+ 10794us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:40.166820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:40.167481 (+ 661us) server_negotiation.cc:407] Connection header received
0809 12:16:40.167685 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:40.167691 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:40.167781 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:40.167909 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:40.169795 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:40.170563 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:40.172658 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:40.173066 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:40.173160 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:40.183369 (+ 10209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:40.183385 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:40.183402 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:40.183434 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:40.186792 (+ 3358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:40.187421 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:40.187428 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:40.187431 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:40.187608 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:40.188004 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:40.188008 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:40.188010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:40.188380 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:40.188670 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:40.188930 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 12:16:40.189109 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10586,"thread_start_us":10766,"threads_started":1}
W20260809 12:16:40.190312 22967 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:42550
W20260809 12:16:40.191104 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 22782: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:16:40.191368 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 22782
I20260809 12:16:40.227238 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 22977
Aug 09 12:16:40 dist-test-slave-39l8 krb5kdc[22631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277797, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.183.194@KRBTEST.COM
I20260809 12:16:40.245699 23591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:40.228078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37084 (local address 127.19.183.194:42423)
0809 12:16:40.228678 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:40.228683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:40.228700 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:16:40.228756 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:40.228760 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:40.228822 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:40.228941 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:40.230562 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:40.231597 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:40.233415 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:40.233964 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:40.234059 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:40.239343 (+ 5284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:40.239358 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:40.239394 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:40.239441 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:40.242644 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:40.243507 (+ 863us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:40.243515 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:40.243517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:40.243661 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:40.244081 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:40.244088 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:40.244095 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:40.244450 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:40.244795 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:40.245074 (+ 279us) server_negotiation.cc:299] Negotiation successful
0809 12:16:40.245310 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":314,"threads_started":1}
W20260809 12:16:40.246419 23151 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:37084
W20260809 12:16:40.247138 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 22977: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:16:40.247311 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 22977
I20260809 12:16:40.281389 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 23164
Aug 09 12:16:40 dist-test-slave-39l8 krb5kdc[22631](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277797, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.19.183.195@KRBTEST.COM
I20260809 12:16:40.296988 23592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:40.282296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38272 (local address 127.19.183.195:45405)
0809 12:16:40.282885 (+ 589us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:40.282890 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:40.282906 (+ 16us) server_negotiation.cc:407] Connection header received
0809 12:16:40.282962 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:40.282966 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:40.283023 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:40.283118 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:40.284442 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:40.285276 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:40.286455 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:40.287012 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:40.287114 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:40.291517 (+ 4403us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:40.291533 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:40.291551 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:40.291611 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:40.294492 (+ 2881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:40.295142 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:40.295150 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:40.295152 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:40.295290 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:40.295641 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:40.295644 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:40.295646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:40.295961 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:40.296187 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:40.296402 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 12:16:40.296559 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":486,"thread_start_us":311,"threads_started":1}
W20260809 12:16:40.297655 23339 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38272
W20260809 12:16:40.298230 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 23164: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:16:40.298375 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 23164
I20260809 12:16:40.328835 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 22646
I20260809 12:16:40.330377 22743 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:16:40.450492 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 22646
2026-08-09T12:16:40Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (9382 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:16:40 dist-test-slave-39l8 krb5kdc[23598](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:16:40 dist-test-slave-39l8 krb5kdc[23598](info): set up 2 sockets
Aug 09 12:16:40 dist-test-slave-39l8 krb5kdc[23598](info): commencing operation
krb5kdc: starting...
W20260809 12:16:41.741248 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.235s user 0.000s sys 0.023s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:16:41 dist-test-slave-39l8 krb5kdc[23598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:16:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:16:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:16:41.960553 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:33963
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:36665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:33963
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:42.293951 23619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:42.294230 23619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:42.294420 23619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:42.301672 23619 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:16:42.301767 23619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:42.301864 23619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:42.301944 23619 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:16:42.302003 23619 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:16:42.312480 23619 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:36665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:33963
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:33963
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:42.313782 23619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:42.315194 23619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:42.326550 23631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:42.327049 23630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:42.327050 23634 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:42.327805 23619 server_base.cc:1034] running on GCE node
I20260809 12:16:42.328639 23619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:42.330077 23619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:42.331357 23619 hybrid_clock.cc:648] HybridClock initialized: now 1786277802331297 us; error 61 us; skew 500 ppm
I20260809 12:16:42.331776 23619 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:42 dist-test-slave-39l8 krb5kdc[23598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277802, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:42.334993 23619 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:16:42.336424 23619 webserver.cc:460] Webserver started at http://127.19.183.254:45303/ using document root <none> and password file <none>
I20260809 12:16:42.337152 23619 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:42.337301 23619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:42.337625 23619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:42.340632 23619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "863cb1b3568341cd83d52b89bbac4bce"
format_stamp: "Formatted at 2026-08-09 12:16:42 on dist-test-slave-39l8"
server_key: "8f3bab12169eb6ea96185325c4cd4a22"
server_key_iv: "2c26d561ba7717c1ae1c17f9391b9545"
server_key_version: "encryptionkey@0"
I20260809 12:16:42.341374 23619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "863cb1b3568341cd83d52b89bbac4bce"
format_stamp: "Formatted at 2026-08-09 12:16:42 on dist-test-slave-39l8"
server_key: "8f3bab12169eb6ea96185325c4cd4a22"
server_key_iv: "2c26d561ba7717c1ae1c17f9391b9545"
server_key_version: "encryptionkey@0"
I20260809 12:16:42.346333 23619 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 12:16:42.349768 23645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:42.351136 23619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:16:42.351323 23619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "863cb1b3568341cd83d52b89bbac4bce"
format_stamp: "Formatted at 2026-08-09 12:16:42 on dist-test-slave-39l8"
server_key: "8f3bab12169eb6ea96185325c4cd4a22"
server_key_iv: "2c26d561ba7717c1ae1c17f9391b9545"
server_key_version: "encryptionkey@0"
I20260809 12:16:42.351537 23619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:42.367089 23619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:42.370949 23619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:42.371261 23619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:42.389966 23619 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:33963
I20260809 12:16:42.389977 23712 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:33963 every 8 connection(s)
I20260809 12:16:42.391434 23619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:16:42.395067 23713 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:42.401064 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 23619
I20260809 12:16:42.401297 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:16:42.401935 20191 external_mini_cluster.cc:1428] Setting key a51181383cb49cc0bc32790feee76008
I20260809 12:16:42.405509 23713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863cb1b3568341cd83d52b89bbac4bce: Bootstrap starting.
I20260809 12:16:42.409215 23713 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 863cb1b3568341cd83d52b89bbac4bce: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:42.410254 23713 log.cc:824] T 00000000000000000000000000000000 P 863cb1b3568341cd83d52b89bbac4bce: Log is configured to *not* fsync() on all Append() calls
I20260809 12:16:42.413160 23713 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 863cb1b3568341cd83d52b89bbac4bce: No bootstrap required, opened a new log
Aug 09 12:16:42 dist-test-slave-39l8 krb5kdc[23598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277801, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:42.421696 23713 raft_consensus.cc:348] T 00000000000000000000000000000000 P 863cb1b3568341cd83d52b89bbac4bce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863cb1b3568341cd83d52b89bbac4bce" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33963 } }
I20260809 12:16:42.422070 23713 raft_consensus.cc:374] T 00000000000000000000000000000000 P 863cb1b3568341cd83d52b89bbac4bce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:42.422184 23713 raft_consensus.cc:729] T 00000000000000000000000000000000 P 863cb1b3568341cd83d52b89bbac4bce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 863cb1b3568341cd83d52b89bbac4bce, State: Initialized, Role: FOLLOWER
I20260809 12:16:42.422878 23713 consensus_queue.cc:260] T 00000000000000000000000000000000 P 863cb1b3568341cd83d52b89bbac4bce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863cb1b3568341cd83d52b89bbac4bce" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33963 } }
I20260809 12:16:42.423194 23713 raft_consensus.cc:388] T 00000000000000000000000000000000 P 863cb1b3568341cd83d52b89bbac4bce [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:16:42.423350 23713 raft_consensus.cc:482] T 00000000000000000000000000000000 P 863cb1b3568341cd83d52b89bbac4bce [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:16:42.423561 23713 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 863cb1b3568341cd83d52b89bbac4bce [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:42.425261 23716 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:42.403904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40826 (local address 127.19.183.254:33963)
0809 12:16:42.404722 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:42.404734 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:42.404772 (+ 38us) server_negotiation.cc:407] Connection header received
0809 12:16:42.406498 (+ 1726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:42.406521 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:42.406962 (+ 441us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:42.407297 (+ 335us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:42.409217 (+ 1920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:42.410549 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:42.412140 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:42.412756 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:42.412877 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:42.417659 (+ 4782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:42.417667 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:42.417685 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:42.417729 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:42.421031 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:42.421705 (+ 674us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:42.421712 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:42.421718 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:42.421866 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:42.422298 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:42.422304 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:42.422306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:42.423041 (+ 735us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:42.423387 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:42.423799 (+ 412us) server_negotiation.cc:299] Negotiation successful
0809 12:16:42.424252 (+ 453us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":383,"threads_started":1}
I20260809 12:16:42.426074 23713 raft_consensus.cc:504] T 00000000000000000000000000000000 P 863cb1b3568341cd83d52b89bbac4bce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863cb1b3568341cd83d52b89bbac4bce" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33963 } }
I20260809 12:16:42.426611 23713 leader_election.cc:302] T 00000000000000000000000000000000 P 863cb1b3568341cd83d52b89bbac4bce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 863cb1b3568341cd83d52b89bbac4bce; no voters:
I20260809 12:16:42.427330 23713 leader_election.cc:288] T 00000000000000000000000000000000 P 863cb1b3568341cd83d52b89bbac4bce [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:16:42.427445 23719 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 863cb1b3568341cd83d52b89bbac4bce [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:16:42.429183 23719 raft_consensus.cc:686] T 00000000000000000000000000000000 P 863cb1b3568341cd83d52b89bbac4bce [term 1 LEADER]: Becoming Leader. State: Replica: 863cb1b3568341cd83d52b89bbac4bce, State: Running, Role: LEADER
I20260809 12:16:42.429510 23713 sys_catalog.cc:564] T 00000000000000000000000000000000 P 863cb1b3568341cd83d52b89bbac4bce [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:16:42.429760 23719 consensus_queue.cc:237] T 00000000000000000000000000000000 P 863cb1b3568341cd83d52b89bbac4bce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863cb1b3568341cd83d52b89bbac4bce" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33963 } }
I20260809 12:16:42.432623 23726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863cb1b3568341cd83d52b89bbac4bce [sys.catalog]: SysCatalogTable state changed. Reason: New leader 863cb1b3568341cd83d52b89bbac4bce. Latest consensus state: current_term: 1 leader_uuid: "863cb1b3568341cd83d52b89bbac4bce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863cb1b3568341cd83d52b89bbac4bce" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33963 } } }
I20260809 12:16:42.432765 23724 sys_catalog.cc:455] T 00000000000000000000000000000000 P 863cb1b3568341cd83d52b89bbac4bce [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "863cb1b3568341cd83d52b89bbac4bce" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "863cb1b3568341cd83d52b89bbac4bce" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33963 } } }
I20260809 12:16:42.433125 23726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863cb1b3568341cd83d52b89bbac4bce [sys.catalog]: This master's current role is: LEADER
I20260809 12:16:42.433197 23724 sys_catalog.cc:458] T 00000000000000000000000000000000 P 863cb1b3568341cd83d52b89bbac4bce [sys.catalog]: This master's current role is: LEADER
I20260809 12:16:42.437915 23738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:16:42.442242 23738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:16:42.451258 23738 catalog_manager.cc:1347] Generated new cluster ID: 9083c2821e71484ab1fb47526ee53814
I20260809 12:16:42.451368 23738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:16:42.482434 23738 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:16:42.484359 23738 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:16:42.501070 23738 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 863cb1b3568341cd83d52b89bbac4bce: Generated new TSK 0
I20260809 12:16:42.502125 23738 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
I20260809 12:16:42.558676 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:33963
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:36665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:42.910279 23756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:42.910563 23756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:42.910812 23756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:42.918334 23756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:42.918453 23756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:42.918658 23756 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:16:42.929834 23756 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:36665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:33963
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:42.931319 23756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:42.932796 23756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:42.946656 23768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:42.946626 23771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:42.946611 23769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:42.947628 23756 server_base.cc:1034] running on GCE node
I20260809 12:16:42.948257 23756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:42.949246 23756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:42.950476 23756 hybrid_clock.cc:648] HybridClock initialized: now 1786277802950435 us; error 39 us; skew 500 ppm
I20260809 12:16:42.950862 23756 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:42 dist-test-slave-39l8 krb5kdc[23598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277802, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:42.954108 23756 init.cc:377] Logged in from keytab as kudu/127.19.183.193@KRBTEST.COM (short username kudu)
I20260809 12:16:42.955752 23756 webserver.cc:460] Webserver started at http://127.19.183.193:43723/ using document root <none> and password file <none>
I20260809 12:16:42.956535 23756 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:42.956699 23756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:42.957029 23756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:42.959959 23756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "33e4340f83854b22804f11a86fb5e5ae"
format_stamp: "Formatted at 2026-08-09 12:16:42 on dist-test-slave-39l8"
server_key: "225505370114f182e66f10cfd4b1df0e"
server_key_iv: "1b3757df788d8e380d24a17b17d3e540"
server_key_version: "encryptionkey@0"
I20260809 12:16:42.960759 23756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "33e4340f83854b22804f11a86fb5e5ae"
format_stamp: "Formatted at 2026-08-09 12:16:42 on dist-test-slave-39l8"
server_key: "225505370114f182e66f10cfd4b1df0e"
server_key_iv: "1b3757df788d8e380d24a17b17d3e540"
server_key_version: "encryptionkey@0"
I20260809 12:16:42.965962 23756 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:16:42.969599 23782 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:42.971055 23756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 12:16:42.971240 23756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "33e4340f83854b22804f11a86fb5e5ae"
format_stamp: "Formatted at 2026-08-09 12:16:42 on dist-test-slave-39l8"
server_key: "225505370114f182e66f10cfd4b1df0e"
server_key_iv: "1b3757df788d8e380d24a17b17d3e540"
server_key_version: "encryptionkey@0"
I20260809 12:16:42.971448 23756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:42.997632 23756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:43.002205 23756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:43.002513 23756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:43.003633 23756 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:16:43.005014 23756 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:43.005087 23756 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:43.005175 23756 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:43.005219 23756 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:43.039908 23756 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:46589
I20260809 12:16:43.040001 23929 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:46589 every 8 connection(s)
I20260809 12:16:43.041283 23756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:16:43.051405 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 23756
I20260809 12:16:43.051690 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:16:43.052464 20191 external_mini_cluster.cc:1428] Setting key 087f2f1d2b3edba8cc453ae5fe9bf524
Aug 09 12:16:43 dist-test-slave-39l8 krb5kdc[23598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277802, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:43.063386 23936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:43.043921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:47553 (local address 127.19.183.254:33963)
0809 12:16:43.044453 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:43.044458 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:43.045242 (+ 784us) server_negotiation.cc:407] Connection header received
0809 12:16:43.047160 (+ 1918us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:43.047166 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:43.047257 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:43.047383 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:43.048975 (+ 1592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:43.049695 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:43.051602 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:43.052081 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:43.052197 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:43.056470 (+ 4273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:43.056478 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:43.056486 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:43.056533 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:43.060186 (+ 3653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:43.060896 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:43.060904 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:43.060910 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:43.061064 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:43.061463 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:43.061470 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:43.061475 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:43.061870 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:43.062016 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:43.062675 (+ 659us) server_negotiation.cc:299] Negotiation successful
0809 12:16:43.062947 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":301,"threads_started":1}
I20260809 12:16:43.064083 23937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:43.044348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:33963 (local address 127.19.183.193:47553)
0809 12:16:43.045060 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:43.045107 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:43.046852 (+ 1745us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:43.047632 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:43.047641 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:43.048166 (+ 525us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:43.048769 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:43.048799 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:43.049851 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:43.049862 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:43.051372 (+ 1510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:43.051400 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:43.052346 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:43.052354 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:43.052700 (+ 346us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:43.053422 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:43.053458 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:43.056249 (+ 2791us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:43.060322 (+ 4073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:43.060329 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:43.060340 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:43.060767 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:43.061173 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:43.061177 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:43.061179 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:43.061348 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:43.061984 (+ 636us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:43.061989 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:43.062301 (+ 312us) client_negotiation.cc:769] Sending connection context
0809 12:16:43.062517 (+ 216us) client_negotiation.cc:240] Negotiation successful
0809 12:16:43.062960 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":275,"threads_started":1}
I20260809 12:16:43.065800 23932 heartbeater.cc:347] Connected to a master server at 127.19.183.254:33963
I20260809 12:16:43.066118 23932 heartbeater.cc:464] Registering TS with master...
I20260809 12:16:43.066901 23932 heartbeater.cc:511] Master 127.19.183.254:33963 requested a full tablet report, sending...
I20260809 12:16:43.069360 23667 ts_manager.cc:194] Registered new tserver with Master: 33e4340f83854b22804f11a86fb5e5ae (127.19.183.193:46589)
I20260809 12:16:43.071583 23667 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:47553
WARNING: no policy specified for kudu/127.19.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
I20260809 12:16:43.106493 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:33963
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:36665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:43.442058 23940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:43.442395 23940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:43.442567 23940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:43.449941 23940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:43.450109 23940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:43.450284 23940 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:16:43.460171 23940 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:36665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:33963
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:43.461472 23940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:43.462759 23940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:43.475858 23949 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:43.475762 23947 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:43.475819 23946 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:43.476725 23940 server_base.cc:1034] running on GCE node
I20260809 12:16:43.477375 23940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:43.478350 23940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:43.479557 23940 hybrid_clock.cc:648] HybridClock initialized: now 1786277803479504 us; error 41 us; skew 500 ppm
I20260809 12:16:43.479884 23940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:43 dist-test-slave-39l8 krb5kdc[23598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277803, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:43.483036 23940 init.cc:377] Logged in from keytab as kudu/127.19.183.194@KRBTEST.COM (short username kudu)
I20260809 12:16:43.484388 23940 webserver.cc:460] Webserver started at http://127.19.183.194:34525/ using document root <none> and password file <none>
I20260809 12:16:43.485021 23940 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:43.485153 23940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:43.485427 23940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:43.488324 23940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "d872de37b05c4aaeaf2f1e3a20844d47"
format_stamp: "Formatted at 2026-08-09 12:16:43 on dist-test-slave-39l8"
server_key: "fd125f4108c3b516e04e739b0d758682"
server_key_iv: "c852de5013361ce816f7ee919f517f36"
server_key_version: "encryptionkey@0"
I20260809 12:16:43.489037 23940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "d872de37b05c4aaeaf2f1e3a20844d47"
format_stamp: "Formatted at 2026-08-09 12:16:43 on dist-test-slave-39l8"
server_key: "fd125f4108c3b516e04e739b0d758682"
server_key_iv: "c852de5013361ce816f7ee919f517f36"
server_key_version: "encryptionkey@0"
I20260809 12:16:43.494001 23940 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:16:43.497378 23956 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:43.498617 23940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:16:43.498812 23940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "d872de37b05c4aaeaf2f1e3a20844d47"
format_stamp: "Formatted at 2026-08-09 12:16:43 on dist-test-slave-39l8"
server_key: "fd125f4108c3b516e04e739b0d758682"
server_key_iv: "c852de5013361ce816f7ee919f517f36"
server_key_version: "encryptionkey@0"
I20260809 12:16:43.499017 23940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:43.528172 23940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:43.532702 23940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:43.533005 23940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:43.534114 23940 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:16:43.535539 23940 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:43.535617 23940 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:43.535776 23940 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:43.535863 23940 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:43.572537 23940 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:39321
I20260809 12:16:43.572619 24115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:39321 every 8 connection(s)
I20260809 12:16:43.574579 23940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:16:43.580127 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 23940
I20260809 12:16:43.580319 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:16:43.580806 20191 external_mini_cluster.cc:1428] Setting key d738756b22e99f3cca6459b1275faca8
Aug 09 12:16:43 dist-test-slave-39l8 krb5kdc[23598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277803, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:43.603823 24118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:43.578005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:53631 (local address 127.19.183.254:33963)
0809 12:16:43.583061 (+ 5056us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:43.583070 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:43.583094 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:16:43.584689 (+ 1595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:43.584694 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:43.584785 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:43.584901 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:43.586973 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:43.588224 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:43.590932 (+ 2708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:43.591755 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:43.591912 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:43.596845 (+ 4933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:43.596854 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:43.596863 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:43.596904 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:43.600260 (+ 3356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:43.601020 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:43.601025 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:43.601027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:43.601148 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:43.601673 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:43.601677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:43.601679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:43.602015 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:43.602264 (+ 249us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:43.603140 (+ 876us) server_negotiation.cc:299] Negotiation successful
0809 12:16:43.603371 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4902,"thread_start_us":4778,"threads_started":1}
I20260809 12:16:43.604943 24119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:43.578572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:33963 (local address 127.19.183.194:53631)
0809 12:16:43.582256 (+ 3684us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:43.582300 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:43.584417 (+ 2117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:43.585361 (+ 944us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:43.585372 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:43.585908 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:43.586760 (+ 852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:43.586790 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:43.588423 (+ 1633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:43.588431 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:43.590775 (+ 2344us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:43.590793 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:43.592037 (+ 1244us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:43.592045 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:43.592394 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:43.593243 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:43.593279 (+ 36us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:43.596525 (+ 3246us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:43.600424 (+ 3899us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:43.600432 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:43.600449 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:43.600871 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:43.601267 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:43.601273 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:43.601276 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:43.601555 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:43.602236 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:43.602245 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:43.602660 (+ 415us) client_negotiation.cc:769] Sending connection context
0809 12:16:43.603030 (+ 370us) client_negotiation.cc:240] Negotiation successful
0809 12:16:43.603598 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3423,"thread_start_us":3218,"threads_started":1}
I20260809 12:16:43.606597 24116 heartbeater.cc:347] Connected to a master server at 127.19.183.254:33963
I20260809 12:16:43.606904 24116 heartbeater.cc:464] Registering TS with master...
I20260809 12:16:43.607720 24116 heartbeater.cc:511] Master 127.19.183.254:33963 requested a full tablet report, sending...
I20260809 12:16:43.609257 23667 ts_manager.cc:194] Registered new tserver with Master: d872de37b05c4aaeaf2f1e3a20844d47 (127.19.183.194:39321)
I20260809 12:16:43.610471 23667 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:53631
WARNING: no policy specified for kudu/127.19.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
I20260809 12:16:43.647957 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:33963
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:36665
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:43.995337 24123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:43.995600 24123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:43.995694 24123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:44.003237 24123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:44.003340 24123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:44.003515 24123 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:16:44.013346 24123 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:36665
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:33963
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:44.015262 24123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:44.017236 24123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:44.032327 24129 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:44.033500 24130 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:44.034875 24132 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:44.035356 24123 server_base.cc:1034] running on GCE node
I20260809 12:16:44.035981 24123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:44.036985 24123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:44.038218 24123 hybrid_clock.cc:648] HybridClock initialized: now 1786277804038155 us; error 57 us; skew 500 ppm
I20260809 12:16:44.038551 24123 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:44 dist-test-slave-39l8 krb5kdc[23598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277804, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:44.042689 24123 init.cc:377] Logged in from keytab as kudu/127.19.183.195@KRBTEST.COM (short username kudu)
I20260809 12:16:44.046270 24123 webserver.cc:460] Webserver started at http://127.19.183.195:45553/ using document root <none> and password file <none>
I20260809 12:16:44.047178 24123 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:44.047330 24123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:44.047642 24123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:44.050534 24123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "d3b345efa8f94ea98bb79cf646c35b73"
format_stamp: "Formatted at 2026-08-09 12:16:44 on dist-test-slave-39l8"
server_key: "54f5ff9175529e888d0e3db20e607fb7"
server_key_iv: "f169bf4f6bed5227d03a35ed44c777f5"
server_key_version: "encryptionkey@0"
I20260809 12:16:44.051433 24123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "d3b345efa8f94ea98bb79cf646c35b73"
format_stamp: "Formatted at 2026-08-09 12:16:44 on dist-test-slave-39l8"
server_key: "54f5ff9175529e888d0e3db20e607fb7"
server_key_iv: "f169bf4f6bed5227d03a35ed44c777f5"
server_key_version: "encryptionkey@0"
I20260809 12:16:44.056697 24123 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:16:44.060300 24142 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:44.061595 24123 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:16:44.061772 24123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "d3b345efa8f94ea98bb79cf646c35b73"
format_stamp: "Formatted at 2026-08-09 12:16:44 on dist-test-slave-39l8"
server_key: "54f5ff9175529e888d0e3db20e607fb7"
server_key_iv: "f169bf4f6bed5227d03a35ed44c777f5"
server_key_version: "encryptionkey@0"
I20260809 12:16:44.061980 24123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:44.074885 23932 heartbeater.cc:503] Master 127.19.183.254:33963 was elected leader, sending a full tablet report...
I20260809 12:16:44.075256 24123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:44.079648 24123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:44.079947 24123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:44.080916 24123 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:16:44.082273 24123 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:44.082384 24123 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:44.082535 24123 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:44.082612 24123 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:44.118234 24123 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:34787
I20260809 12:16:44.118357 24293 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:34787 every 8 connection(s)
I20260809 12:16:44.119652 24123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:16:44.128750 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 24123
I20260809 12:16:44.128970 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:16:44.129490 20191 external_mini_cluster.cc:1428] Setting key 7edfd5bb5f78b4a2a7241798244a559d
Aug 09 12:16:44 dist-test-slave-39l8 krb5kdc[23598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277804, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:44.140734 24296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:44.122262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:46851 (local address 127.19.183.254:33963)
0809 12:16:44.122798 (+ 536us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:44.122803 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:44.123851 (+ 1048us) server_negotiation.cc:407] Connection header received
0809 12:16:44.125932 (+ 2081us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:44.125938 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:44.126028 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:44.126145 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:44.127803 (+ 1658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:44.128789 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:44.130609 (+ 1820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:44.131179 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:44.131278 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:44.135239 (+ 3961us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:44.135243 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:44.135247 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:44.135280 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:44.137806 (+ 2526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:44.138637 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:44.138640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:44.138643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:44.138825 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:44.139254 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:44.139257 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:44.139259 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:44.139517 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:44.139652 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:44.140281 (+ 629us) server_negotiation.cc:299] Negotiation successful
0809 12:16:44.140462 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":234,"threads_started":1}
I20260809 12:16:44.141784 24297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:44.122952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:33963 (local address 127.19.183.195:46851)
0809 12:16:44.123672 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:44.123719 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:44.125645 (+ 1926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:44.126410 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:44.126423 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:44.127047 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:44.127624 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:44.127642 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:44.128978 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:44.128983 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:44.130427 (+ 1444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:44.130441 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:44.131413 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:44.131421 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:44.131632 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:44.132496 (+ 864us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:44.132525 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:44.135043 (+ 2518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:44.137954 (+ 2911us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:44.137982 (+ 28us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:44.137999 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:44.138516 (+ 517us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:44.138936 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:44.138939 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:44.138941 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:44.139133 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:44.139647 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:44.139652 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:44.139937 (+ 285us) client_negotiation.cc:769] Sending connection context
0809 12:16:44.140179 (+ 242us) client_negotiation.cc:240] Negotiation successful
0809 12:16:44.140575 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":498,"thread_start_us":279,"threads_started":1}
I20260809 12:16:44.143241 24294 heartbeater.cc:347] Connected to a master server at 127.19.183.254:33963
I20260809 12:16:44.143551 24294 heartbeater.cc:464] Registering TS with master...
I20260809 12:16:44.144330 24294 heartbeater.cc:511] Master 127.19.183.254:33963 requested a full tablet report, sending...
I20260809 12:16:44.146090 23667 ts_manager.cc:194] Registered new tserver with Master: d3b345efa8f94ea98bb79cf646c35b73 (127.19.183.195:34787)
I20260809 12:16:44.146948 23667 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:46851
I20260809 12:16:44.158181 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 12:16:44.198885 24296 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:16:44.196442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40832 (local address 127.19.183.254:33963)
0809 12:16:44.196636 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:44.196641 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:44.197036 (+ 395us) server_negotiation.cc:407] Connection header received
0809 12:16:44.197197 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:44.197202 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:44.197286 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:44.197384 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:44.198559 (+ 1175us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40832: BlockingRecv error: recv got EOF from 127.0.0.1:40832 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:16:44.202606 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 23756
W20260809 12:16:44.205542 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 23756: Not authorized: Client connection negotiation failed: client connection to 127.19.183.193:46589: server requires authentication, but client does not have Kerberos credentials available
I20260809 12:16:44.205760 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 23756
I20260809 12:16:44.229461 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 23940
W20260809 12:16:44.232487 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 23940: Not authorized: Client connection negotiation failed: client connection to 127.19.183.194:39321: server requires authentication, but client does not have Kerberos credentials available
I20260809 12:16:44.232648 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 23940
I20260809 12:16:44.257131 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 24123
W20260809 12:16:44.260211 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 24123: Not authorized: Client connection negotiation failed: client connection to 127.19.183.195:34787: server requires authentication, but client does not have Kerberos credentials available
I20260809 12:16:44.260336 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 24123
I20260809 12:16:44.285784 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 23619
I20260809 12:16:44.287047 23708 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:16:44.391858 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 23619
2026-08-09T12:16:44Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (3940 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:16:44 dist-test-slave-39l8 krb5kdc[24317](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:16:44 dist-test-slave-39l8 krb5kdc[24317](info): set up 2 sockets
Aug 09 12:16:44 dist-test-slave-39l8 krb5kdc[24317](info): commencing operation
krb5kdc: starting...
W20260809 12:16:45.664671 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.218s user 0.005s sys 0.022s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:16:45 dist-test-slave-39l8 krb5kdc[24317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:16:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:16:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:16:45.884176 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:34705
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:37677
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:34705
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:46.220557 24331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:46.220856 24331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:46.220950 24331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:46.228060 24331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:16:46.228163 24331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:46.228210 24331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:46.228267 24331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:16:46.228317 24331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:16:46.237912 24331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:37677
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:34705
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:34705
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:46.239164 24331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:46.240386 24331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:46.252255 24340 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:46.252300 24341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:46.252871 24344 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:46.253173 24331 server_base.cc:1034] running on GCE node
I20260809 12:16:46.253943 24331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:46.255398 24331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:46.256680 24331 hybrid_clock.cc:648] HybridClock initialized: now 1786277806256607 us; error 66 us; skew 500 ppm
I20260809 12:16:46.257035 24331 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:46 dist-test-slave-39l8 krb5kdc[24317](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277806, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:46.264375 24331 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:16:46.265949 24331 webserver.cc:460] Webserver started at http://127.19.183.254:36975/ using document root <none> and password file <none>
I20260809 12:16:46.266597 24331 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:46.266749 24331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:46.267043 24331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:46.269937 24331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "35a77a3a222b43cebbbced0ba81db048"
format_stamp: "Formatted at 2026-08-09 12:16:46 on dist-test-slave-39l8"
server_key: "7f702a42b9e8964d5b7cccd91b2d4f42"
server_key_iv: "e3b9b6d83a3ba3a24a084ec8bbab0a86"
server_key_version: "encryptionkey@0"
I20260809 12:16:46.270658 24331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "35a77a3a222b43cebbbced0ba81db048"
format_stamp: "Formatted at 2026-08-09 12:16:46 on dist-test-slave-39l8"
server_key: "7f702a42b9e8964d5b7cccd91b2d4f42"
server_key_iv: "e3b9b6d83a3ba3a24a084ec8bbab0a86"
server_key_version: "encryptionkey@0"
I20260809 12:16:46.275728 24331 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:16:46.279016 24354 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:46.280342 24331 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:16:46.280517 24331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "35a77a3a222b43cebbbced0ba81db048"
format_stamp: "Formatted at 2026-08-09 12:16:46 on dist-test-slave-39l8"
server_key: "7f702a42b9e8964d5b7cccd91b2d4f42"
server_key_iv: "e3b9b6d83a3ba3a24a084ec8bbab0a86"
server_key_version: "encryptionkey@0"
I20260809 12:16:46.280711 24331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:46.299327 24331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:46.303156 24331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:46.303429 24331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:46.322041 24331 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:34705
I20260809 12:16:46.322079 24426 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:34705 every 8 connection(s)
I20260809 12:16:46.323415 24331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:16:46.326980 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 24331
I20260809 12:16:46.327095 24428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:46.327215 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:16:46.327913 20191 external_mini_cluster.cc:1428] Setting key 555a006893c2bc677156e6f331076568
I20260809 12:16:46.340832 24428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35a77a3a222b43cebbbced0ba81db048: Bootstrap starting.
Aug 09 12:16:46 dist-test-slave-39l8 krb5kdc[24317](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277805, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:46.344568 24428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 35a77a3a222b43cebbbced0ba81db048: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:46.345826 24428 log.cc:824] T 00000000000000000000000000000000 P 35a77a3a222b43cebbbced0ba81db048: Log is configured to *not* fsync() on all Append() calls
I20260809 12:16:46.348775 24428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 35a77a3a222b43cebbbced0ba81db048: No bootstrap required, opened a new log
I20260809 12:16:46.352247 24432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:46.329672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38618 (local address 127.19.183.254:34705)
0809 12:16:46.330557 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:46.330568 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:46.330609 (+ 41us) server_negotiation.cc:407] Connection header received
0809 12:16:46.333067 (+ 2458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:46.333089 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:46.333448 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:46.333777 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:46.335641 (+ 1864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:46.336880 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:46.338783 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:46.339409 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:46.339534 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:46.344287 (+ 4753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:46.344295 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:46.344309 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:46.344347 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:46.347988 (+ 3641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:46.348708 (+ 720us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:46.348714 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:46.348721 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:46.348998 (+ 277us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:46.349416 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:46.349422 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:46.349425 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:46.349979 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:46.350311 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:46.350859 (+ 548us) server_negotiation.cc:299] Negotiation successful
0809 12:16:46.351261 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":676,"thread_start_us":445,"threads_started":1}
I20260809 12:16:46.356148 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:16:46.357048 24428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 35a77a3a222b43cebbbced0ba81db048 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35a77a3a222b43cebbbced0ba81db048" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34705 } }
I20260809 12:16:46.357404 24428 raft_consensus.cc:374] T 00000000000000000000000000000000 P 35a77a3a222b43cebbbced0ba81db048 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:46.357522 24428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 35a77a3a222b43cebbbced0ba81db048 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35a77a3a222b43cebbbced0ba81db048, State: Initialized, Role: FOLLOWER
I20260809 12:16:46.358065 24428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 35a77a3a222b43cebbbced0ba81db048 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35a77a3a222b43cebbbced0ba81db048" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34705 } }
I20260809 12:16:46.358291 24428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 35a77a3a222b43cebbbced0ba81db048 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:16:46.358429 24428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 35a77a3a222b43cebbbced0ba81db048 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:16:46.358620 24428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 35a77a3a222b43cebbbced0ba81db048 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:46.360776 24428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 35a77a3a222b43cebbbced0ba81db048 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35a77a3a222b43cebbbced0ba81db048" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34705 } }
I20260809 12:16:46.361274 24428 leader_election.cc:302] T 00000000000000000000000000000000 P 35a77a3a222b43cebbbced0ba81db048 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 35a77a3a222b43cebbbced0ba81db048; no voters:
I20260809 12:16:46.361945 24428 leader_election.cc:288] T 00000000000000000000000000000000 P 35a77a3a222b43cebbbced0ba81db048 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:16:46.362059 24434 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 35a77a3a222b43cebbbced0ba81db048 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:16:46.363417 24434 raft_consensus.cc:686] T 00000000000000000000000000000000 P 35a77a3a222b43cebbbced0ba81db048 [term 1 LEADER]: Becoming Leader. State: Replica: 35a77a3a222b43cebbbced0ba81db048, State: Running, Role: LEADER
I20260809 12:16:46.363963 24434 consensus_queue.cc:237] T 00000000000000000000000000000000 P 35a77a3a222b43cebbbced0ba81db048 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35a77a3a222b43cebbbced0ba81db048" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34705 } }
I20260809 12:16:46.364358 24428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 35a77a3a222b43cebbbced0ba81db048 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:16:46.366981 24435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35a77a3a222b43cebbbced0ba81db048 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "35a77a3a222b43cebbbced0ba81db048" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35a77a3a222b43cebbbced0ba81db048" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34705 } } }
I20260809 12:16:46.367453 24435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35a77a3a222b43cebbbced0ba81db048 [sys.catalog]: This master's current role is: LEADER
I20260809 12:16:46.372025 24436 sys_catalog.cc:455] T 00000000000000000000000000000000 P 35a77a3a222b43cebbbced0ba81db048 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 35a77a3a222b43cebbbced0ba81db048. Latest consensus state: current_term: 1 leader_uuid: "35a77a3a222b43cebbbced0ba81db048" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "35a77a3a222b43cebbbced0ba81db048" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34705 } } }
I20260809 12:16:46.372397 24436 sys_catalog.cc:458] T 00000000000000000000000000000000 P 35a77a3a222b43cebbbced0ba81db048 [sys.catalog]: This master's current role is: LEADER
I20260809 12:16:46.379904 24443 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:16:46.386552 24443 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:16:46.405421 24443 catalog_manager.cc:1347] Generated new cluster ID: 3db4b64298ce4d59a5d744de9c7edb64
I20260809 12:16:46.405551 24443 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:16:46.419831 24432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:46.410501 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38622 (local address 127.19.183.254:34705)
0809 12:16:46.410677 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:46.410683 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:46.411287 (+ 604us) server_negotiation.cc:407] Connection header received
0809 12:16:46.411471 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:46.411478 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:46.411566 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:46.411701 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:46.412902 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:46.414047 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:46.415850 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:46.416580 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:46.416722 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:46.417926 (+ 1204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:46.417933 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:46.417938 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:16:46.418992 (+ 1054us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:46.419059 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:16:46.419096 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:46.419221 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:46.419354 (+ 133us) server_negotiation.cc:299] Negotiation successful
0809 12:16:46.419456 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 12:16:46.427839 24443 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:16:46.429162 24443 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:16:46.437870 24443 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 35a77a3a222b43cebbbced0ba81db048: Generated new TSK 0
I20260809 12:16:46.438687 24443 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:16:46.447687 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 24331
I20260809 12:16:46.449452 24421 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:16:46.540674 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 24331
2026-08-09T12:16:46Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (2140 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:16:46 dist-test-slave-39l8 krb5kdc[24482](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:16:46 dist-test-slave-39l8 krb5kdc[24482](info): set up 2 sockets
Aug 09 12:16:46 dist-test-slave-39l8 krb5kdc[24482](info): commencing operation
krb5kdc: starting...
W20260809 12:16:47.830418 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.244s user 0.005s sys 0.021s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:16:47 dist-test-slave-39l8 krb5kdc[24482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:16:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:16:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:16:48.052336 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:41455
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:41455
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:48.385116 24501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:48.385407 24501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:48.385588 24501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:48.392652 24501 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:16:48.392781 24501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:48.392875 24501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:48.392951 24501 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:16:48.392995 24501 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:16:48.402168 24501 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:41455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:41455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:48.403417 24501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:48.404667 24501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:48.416474 24506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:48.416394 24510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:48.416507 24507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:48.417385 24501 server_base.cc:1034] running on GCE node
I20260809 12:16:48.418205 24501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:48.419606 24501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:48.420804 24501 hybrid_clock.cc:648] HybridClock initialized: now 1786277808420769 us; error 23 us; skew 500 ppm
I20260809 12:16:48.421145 24501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:48 dist-test-slave-39l8 krb5kdc[24482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277808, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:48.424211 24501 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:16:48.425587 24501 webserver.cc:460] Webserver started at http://127.19.183.254:39193/ using document root <none> and password file <none>
I20260809 12:16:48.426258 24501 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:48.426391 24501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:48.426782 24501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:48.429747 24501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "576df3f0314141a18ca2094119fa6c3b"
format_stamp: "Formatted at 2026-08-09 12:16:48 on dist-test-slave-39l8"
server_key: "47d08bdb0dadb88a41936878c568bd51"
server_key_iv: "9599a50b1d93ce224370a1545bef786c"
server_key_version: "encryptionkey@0"
I20260809 12:16:48.430514 24501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "576df3f0314141a18ca2094119fa6c3b"
format_stamp: "Formatted at 2026-08-09 12:16:48 on dist-test-slave-39l8"
server_key: "47d08bdb0dadb88a41936878c568bd51"
server_key_iv: "9599a50b1d93ce224370a1545bef786c"
server_key_version: "encryptionkey@0"
I20260809 12:16:48.435672 24501 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 12:16:48.439029 24523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:48.440239 24501 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:16:48.440424 24501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "576df3f0314141a18ca2094119fa6c3b"
format_stamp: "Formatted at 2026-08-09 12:16:48 on dist-test-slave-39l8"
server_key: "47d08bdb0dadb88a41936878c568bd51"
server_key_iv: "9599a50b1d93ce224370a1545bef786c"
server_key_version: "encryptionkey@0"
I20260809 12:16:48.440619 24501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:48.454005 24501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:48.457778 24501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:48.458069 24501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:48.477160 24501 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:41455
I20260809 12:16:48.477183 24591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:41455 every 8 connection(s)
I20260809 12:16:48.478609 24501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:16:48.482301 24593 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:48.484568 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 24501
I20260809 12:16:48.484788 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:16:48.485556 20191 external_mini_cluster.cc:1428] Setting key 6dfaa1f1278792a06bb94252ef42977b
I20260809 12:16:48.495206 24593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 576df3f0314141a18ca2094119fa6c3b: Bootstrap starting.
I20260809 12:16:48.499089 24593 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 576df3f0314141a18ca2094119fa6c3b: Neither blocks nor log segments found. Creating new log.
Aug 09 12:16:48 dist-test-slave-39l8 krb5kdc[24482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277807, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:48.500313 24593 log.cc:824] T 00000000000000000000000000000000 P 576df3f0314141a18ca2094119fa6c3b: Log is configured to *not* fsync() on all Append() calls
I20260809 12:16:48.503933 24593 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 576df3f0314141a18ca2094119fa6c3b: No bootstrap required, opened a new log
I20260809 12:16:48.510074 24597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:48.487703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53392 (local address 127.19.183.254:41455)
0809 12:16:48.488543 (+ 840us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:48.488554 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:48.488600 (+ 46us) server_negotiation.cc:407] Connection header received
0809 12:16:48.490214 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:48.490235 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:48.490598 (+ 363us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:48.490965 (+ 367us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:48.492711 (+ 1746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:48.494000 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:48.495830 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:48.496464 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:48.496597 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:48.501502 (+ 4905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:48.501510 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:48.501523 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:48.501559 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:48.505543 (+ 3984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:48.506205 (+ 662us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:48.506214 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:48.506222 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:48.506449 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:48.506872 (+ 423us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:48.506879 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:48.506884 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:48.507611 (+ 727us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:48.507964 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:48.508271 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 12:16:48.508698 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":620,"thread_start_us":372,"threads_started":1}
I20260809 12:16:48.513862 24593 raft_consensus.cc:348] T 00000000000000000000000000000000 P 576df3f0314141a18ca2094119fa6c3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "576df3f0314141a18ca2094119fa6c3b" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41455 } }
I20260809 12:16:48.514223 24593 raft_consensus.cc:374] T 00000000000000000000000000000000 P 576df3f0314141a18ca2094119fa6c3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:48.514355 24593 raft_consensus.cc:729] T 00000000000000000000000000000000 P 576df3f0314141a18ca2094119fa6c3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 576df3f0314141a18ca2094119fa6c3b, State: Initialized, Role: FOLLOWER
I20260809 12:16:48.514982 24593 consensus_queue.cc:260] T 00000000000000000000000000000000 P 576df3f0314141a18ca2094119fa6c3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "576df3f0314141a18ca2094119fa6c3b" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41455 } }
I20260809 12:16:48.515215 24593 raft_consensus.cc:388] T 00000000000000000000000000000000 P 576df3f0314141a18ca2094119fa6c3b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:16:48.515365 24593 raft_consensus.cc:482] T 00000000000000000000000000000000 P 576df3f0314141a18ca2094119fa6c3b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:16:48.515542 24593 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 576df3f0314141a18ca2094119fa6c3b [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:48.517781 24593 raft_consensus.cc:504] T 00000000000000000000000000000000 P 576df3f0314141a18ca2094119fa6c3b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "576df3f0314141a18ca2094119fa6c3b" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41455 } }
I20260809 12:16:48.518266 24593 leader_election.cc:302] T 00000000000000000000000000000000 P 576df3f0314141a18ca2094119fa6c3b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 576df3f0314141a18ca2094119fa6c3b; no voters:
I20260809 12:16:48.518910 24593 leader_election.cc:288] T 00000000000000000000000000000000 P 576df3f0314141a18ca2094119fa6c3b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:16:48.519050 24600 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 576df3f0314141a18ca2094119fa6c3b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:16:48.519747 24600 raft_consensus.cc:686] T 00000000000000000000000000000000 P 576df3f0314141a18ca2094119fa6c3b [term 1 LEADER]: Becoming Leader. State: Replica: 576df3f0314141a18ca2094119fa6c3b, State: Running, Role: LEADER
I20260809 12:16:48.520337 24600 consensus_queue.cc:237] T 00000000000000000000000000000000 P 576df3f0314141a18ca2094119fa6c3b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "576df3f0314141a18ca2094119fa6c3b" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41455 } }
I20260809 12:16:48.521162 24593 sys_catalog.cc:564] T 00000000000000000000000000000000 P 576df3f0314141a18ca2094119fa6c3b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:16:48.523274 24601 sys_catalog.cc:455] T 00000000000000000000000000000000 P 576df3f0314141a18ca2094119fa6c3b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "576df3f0314141a18ca2094119fa6c3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "576df3f0314141a18ca2094119fa6c3b" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41455 } } }
I20260809 12:16:48.523686 24601 sys_catalog.cc:458] T 00000000000000000000000000000000 P 576df3f0314141a18ca2094119fa6c3b [sys.catalog]: This master's current role is: LEADER
I20260809 12:16:48.524454 24602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 576df3f0314141a18ca2094119fa6c3b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 576df3f0314141a18ca2094119fa6c3b. Latest consensus state: current_term: 1 leader_uuid: "576df3f0314141a18ca2094119fa6c3b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "576df3f0314141a18ca2094119fa6c3b" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41455 } } }
I20260809 12:16:48.524798 24602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 576df3f0314141a18ca2094119fa6c3b [sys.catalog]: This master's current role is: LEADER
I20260809 12:16:48.528002 24612 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:16:48.533321 24612 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:16:48.541590 24612 catalog_manager.cc:1347] Generated new cluster ID: 35db4da98bc640c2a9f05858a2c37ff2
I20260809 12:16:48.541705 24612 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:16:48.559446 24612 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:16:48.560957 24612 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:16:48.574551 24612 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 576df3f0314141a18ca2094119fa6c3b: Generated new TSK 0
I20260809 12:16:48.575426 24612 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
I20260809 12:16:48.632526 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:41455
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:48.976591 24628 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:48.976876 24628 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:48.976970 24628 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:48.984006 24628 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:48.984108 24628 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:48.984328 24628 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:16:48.993896 24628 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:41455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:48.995253 24628 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:48.996654 24628 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:49.010056 24634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:49.010038 24635 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:49.010298 24637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:49.011343 24628 server_base.cc:1034] running on GCE node
I20260809 12:16:49.012194 24628 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:49.013216 24628 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:49.014492 24628 hybrid_clock.cc:648] HybridClock initialized: now 1786277809014447 us; error 29 us; skew 500 ppm
I20260809 12:16:49.014909 24628 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:49 dist-test-slave-39l8 krb5kdc[24482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277809, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:49.018275 24628 init.cc:377] Logged in from keytab as kudu/127.19.183.193@KRBTEST.COM (short username kudu)
I20260809 12:16:49.019802 24628 webserver.cc:460] Webserver started at http://127.19.183.193:39691/ using document root <none> and password file <none>
I20260809 12:16:49.020570 24628 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:49.020730 24628 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:49.021060 24628 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:49.024210 24628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "e5ee84c1f55444528631046dae769bfe"
format_stamp: "Formatted at 2026-08-09 12:16:49 on dist-test-slave-39l8"
server_key: "c8ab69beae3d6bf43472cfa1ba5240e8"
server_key_iv: "fa9905f04e2409b9fec95ffafdab1be2"
server_key_version: "encryptionkey@0"
I20260809 12:16:49.025198 24628 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "e5ee84c1f55444528631046dae769bfe"
format_stamp: "Formatted at 2026-08-09 12:16:49 on dist-test-slave-39l8"
server_key: "c8ab69beae3d6bf43472cfa1ba5240e8"
server_key_iv: "fa9905f04e2409b9fec95ffafdab1be2"
server_key_version: "encryptionkey@0"
I20260809 12:16:49.030838 24628 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.006s
I20260809 12:16:49.034314 24645 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:49.035869 24628 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 12:16:49.036069 24628 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "e5ee84c1f55444528631046dae769bfe"
format_stamp: "Formatted at 2026-08-09 12:16:49 on dist-test-slave-39l8"
server_key: "c8ab69beae3d6bf43472cfa1ba5240e8"
server_key_iv: "fa9905f04e2409b9fec95ffafdab1be2"
server_key_version: "encryptionkey@0"
I20260809 12:16:49.036306 24628 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:49.051975 24628 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:49.056457 24628 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:49.056777 24628 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:49.057780 24628 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:16:49.059165 24628 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:49.059239 24628 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:49.059427 24628 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:49.059530 24628 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:49.093331 24628 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:33853
I20260809 12:16:49.093367 24805 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:33853 every 8 connection(s)
I20260809 12:16:49.094774 24628 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:16:49.096787 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 24628
I20260809 12:16:49.097007 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:16:49.097787 20191 external_mini_cluster.cc:1428] Setting key e2814394841741de1e58e58b90786ac2
Aug 09 12:16:49 dist-test-slave-39l8 krb5kdc[24482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277809, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:49.116051 24808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:49.097714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:40547 (local address 127.19.183.254:41455)
0809 12:16:49.098232 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:49.098240 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:49.099319 (+ 1079us) server_negotiation.cc:407] Connection header received
0809 12:16:49.101211 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:49.101216 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:49.101325 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:49.101448 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:49.103081 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:49.103805 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:49.105705 (+ 1900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:49.106115 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:49.106264 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:49.110085 (+ 3821us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:49.110091 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:49.110096 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:49.110141 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:49.112840 (+ 2699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:49.113594 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:49.113598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:49.113600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:49.113716 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:49.114216 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:49.114220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:49.114222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:49.114490 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:49.114657 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:49.115430 (+ 773us) server_negotiation.cc:299] Negotiation successful
0809 12:16:49.115665 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":373,"thread_start_us":252,"threads_started":1}
I20260809 12:16:49.117017 24809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:49.098287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:41455 (local address 127.19.183.193:40547)
0809 12:16:49.099144 (+ 857us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:49.099191 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:49.100943 (+ 1752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:49.101714 (+ 771us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:49.101722 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:49.102220 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:49.102893 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:49.102914 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:49.103974 (+ 1060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:49.103984 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:49.105543 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:49.105557 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:49.106359 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:49.106368 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:49.106624 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:49.107374 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:49.107397 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:49.109863 (+ 2466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:49.112990 (+ 3127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:49.113009 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:49.113024 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:49.113451 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:49.113834 (+ 383us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:49.113851 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:49.113855 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:49.114065 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:49.114633 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:49.114640 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:49.115009 (+ 369us) client_negotiation.cc:769] Sending connection context
0809 12:16:49.115307 (+ 298us) client_negotiation.cc:240] Negotiation successful
0809 12:16:49.115844 (+ 537us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"thread_start_us":367,"threads_started":1}
I20260809 12:16:49.118882 24806 heartbeater.cc:347] Connected to a master server at 127.19.183.254:41455
I20260809 12:16:49.119216 24806 heartbeater.cc:464] Registering TS with master...
I20260809 12:16:49.119977 24806 heartbeater.cc:511] Master 127.19.183.254:41455 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.19.183.194@KRBTEST.COM; defaulting to no policy
I20260809 12:16:49.122277 24547 ts_manager.cc:194] Registered new tserver with Master: e5ee84c1f55444528631046dae769bfe (127.19.183.193:33853)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.194@KRBTEST.COM" created.
I20260809 12:16:49.123924 24547 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:40547
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
I20260809 12:16:49.148005 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:41455
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:49.501358 24813 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:49.501619 24813 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:49.501814 24813 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:49.509137 24813 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:49.509241 24813 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:49.509452 24813 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:16:49.519073 24813 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:41455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:49.520392 24813 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:49.521860 24813 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:49.536536 24822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:49.536638 24823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:49.537307 24826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:49.538625 24813 server_base.cc:1034] running on GCE node
I20260809 12:16:49.539453 24813 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:49.540777 24813 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:49.542042 24813 hybrid_clock.cc:648] HybridClock initialized: now 1786277809541911 us; error 99 us; skew 500 ppm
I20260809 12:16:49.542497 24813 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:49 dist-test-slave-39l8 krb5kdc[24482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277809, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:49.546635 24813 init.cc:377] Logged in from keytab as kudu/127.19.183.194@KRBTEST.COM (short username kudu)
I20260809 12:16:49.548233 24813 webserver.cc:460] Webserver started at http://127.19.183.194:45899/ using document root <none> and password file <none>
I20260809 12:16:49.548971 24813 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:49.549121 24813 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:49.549430 24813 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:49.552520 24813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "24ccc93e81e147cea2152d3d14011270"
format_stamp: "Formatted at 2026-08-09 12:16:49 on dist-test-slave-39l8"
server_key: "9e4a4cbc1df85295b5886b9d6c7dabfe"
server_key_iv: "9571cd6f5fd634dc0038bea91b7ced99"
server_key_version: "encryptionkey@0"
I20260809 12:16:49.553339 24813 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "24ccc93e81e147cea2152d3d14011270"
format_stamp: "Formatted at 2026-08-09 12:16:49 on dist-test-slave-39l8"
server_key: "9e4a4cbc1df85295b5886b9d6c7dabfe"
server_key_iv: "9571cd6f5fd634dc0038bea91b7ced99"
server_key_version: "encryptionkey@0"
I20260809 12:16:49.558794 24813 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 12:16:49.562505 24836 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:49.564103 24813 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:16:49.564284 24813 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "24ccc93e81e147cea2152d3d14011270"
format_stamp: "Formatted at 2026-08-09 12:16:49 on dist-test-slave-39l8"
server_key: "9e4a4cbc1df85295b5886b9d6c7dabfe"
server_key_iv: "9571cd6f5fd634dc0038bea91b7ced99"
server_key_version: "encryptionkey@0"
I20260809 12:16:49.564509 24813 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:49.587559 24813 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:49.592176 24813 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:49.592502 24813 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:49.593581 24813 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:16:49.595162 24813 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:49.595270 24813 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:49.595422 24813 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:49.595498 24813 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:49.635249 24813 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:43777
I20260809 12:16:49.635283 24998 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:43777 every 8 connection(s)
I20260809 12:16:49.636548 24813 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:16:49.644438 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 24813
I20260809 12:16:49.644635 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:16:49.645298 20191 external_mini_cluster.cc:1428] Setting key b460669637d278bf9fa241b7465781d4
Aug 09 12:16:49 dist-test-slave-39l8 krb5kdc[24482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277809, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:49.663924 25002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:49.639394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:54205 (local address 127.19.183.254:41455)
0809 12:16:49.639900 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:49.639907 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:49.641352 (+ 1445us) server_negotiation.cc:407] Connection header received
0809 12:16:49.645079 (+ 3727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:49.645087 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:49.645194 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:49.645360 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:49.647929 (+ 2569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:49.648976 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:49.652094 (+ 3118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:49.652845 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:49.652996 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:49.657963 (+ 4967us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:49.657968 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:49.657972 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:49.658014 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:49.660733 (+ 2719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:49.661492 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:49.661502 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:49.661504 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:49.661630 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:49.662105 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:49.662109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:49.662111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:49.662375 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:49.662538 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:49.663330 (+ 792us) server_negotiation.cc:299] Negotiation successful
0809 12:16:49.663554 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":245,"threads_started":1}
I20260809 12:16:49.665076 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:49.639962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:41455 (local address 127.19.183.194:54205)
0809 12:16:49.641086 (+ 1124us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:49.641172 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:49.644703 (+ 3531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:49.645768 (+ 1065us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:49.645780 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:49.646610 (+ 830us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:49.647592 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:49.647623 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:49.649162 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:49.649172 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:49.651881 (+ 2709us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:49.651904 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:49.653123 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:49.653133 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:49.653519 (+ 386us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:49.654504 (+ 985us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:49.654557 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:49.657698 (+ 3141us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:49.660886 (+ 3188us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:49.660895 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:49.660912 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:49.661353 (+ 441us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:49.661752 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:49.661757 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:49.661761 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:49.661963 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:49.662530 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:49.662537 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:49.662952 (+ 415us) client_negotiation.cc:769] Sending connection context
0809 12:16:49.663221 (+ 269us) client_negotiation.cc:240] Negotiation successful
0809 12:16:49.663778 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":721,"thread_start_us":434,"threads_started":1}
I20260809 12:16:49.666774 24999 heartbeater.cc:347] Connected to a master server at 127.19.183.254:41455
I20260809 12:16:49.667111 24999 heartbeater.cc:464] Registering TS with master...
I20260809 12:16:49.668108 24999 heartbeater.cc:511] Master 127.19.183.254:41455 requested a full tablet report, sending...
I20260809 12:16:49.669750 24547 ts_manager.cc:194] Registered new tserver with Master: 24ccc93e81e147cea2152d3d14011270 (127.19.183.194:43777)
I20260809 12:16:49.670609 24547 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:54205
WARNING: no policy specified for kudu/127.19.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
I20260809 12:16:49.704918 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:41455
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40863
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:50.044690 25006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:50.044986 25006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:50.045135 25006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:50.052350 25006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:50.052448 25006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:50.052558 25006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:16:50.061837 25006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40863
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:41455
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:50.063030 25006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:50.064248 25006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:50.077159 25014 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:50.077220 25012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:50.077251 25017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:50.080101 25006 server_base.cc:1034] running on GCE node
I20260809 12:16:50.080747 25006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:50.081689 25006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:50.082917 25006 hybrid_clock.cc:648] HybridClock initialized: now 1786277810082876 us; error 53 us; skew 500 ppm
I20260809 12:16:50.083242 25006 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:50 dist-test-slave-39l8 krb5kdc[24482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277810, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:50.086489 25006 init.cc:377] Logged in from keytab as kudu/127.19.183.195@KRBTEST.COM (short username kudu)
I20260809 12:16:50.088107 25006 webserver.cc:460] Webserver started at http://127.19.183.195:33025/ using document root <none> and password file <none>
I20260809 12:16:50.088889 25006 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:50.089043 25006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:50.089360 25006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:50.092480 25006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "1265a494f5724aa9bc0bdcb02b772967"
format_stamp: "Formatted at 2026-08-09 12:16:50 on dist-test-slave-39l8"
server_key: "1d491a4a68f1c46e4b3fc15f6338c922"
server_key_iv: "ee866d75664bb41811866dbe7e60f47d"
server_key_version: "encryptionkey@0"
I20260809 12:16:50.093297 25006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "1265a494f5724aa9bc0bdcb02b772967"
format_stamp: "Formatted at 2026-08-09 12:16:50 on dist-test-slave-39l8"
server_key: "1d491a4a68f1c46e4b3fc15f6338c922"
server_key_iv: "ee866d75664bb41811866dbe7e60f47d"
server_key_version: "encryptionkey@0"
I20260809 12:16:50.098480 25006 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 12:16:50.101779 25025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:50.103156 25006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:16:50.103330 25006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "1265a494f5724aa9bc0bdcb02b772967"
format_stamp: "Formatted at 2026-08-09 12:16:50 on dist-test-slave-39l8"
server_key: "1d491a4a68f1c46e4b3fc15f6338c922"
server_key_iv: "ee866d75664bb41811866dbe7e60f47d"
server_key_version: "encryptionkey@0"
I20260809 12:16:50.103533 25006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:50.114861 25006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:50.118520 25006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:50.118893 25006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:50.119804 25006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:16:50.121091 25006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:50.121197 25006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:50.121356 25006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:50.121428 25006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:50.126978 24806 heartbeater.cc:503] Master 127.19.183.254:41455 was elected leader, sending a full tablet report...
I20260809 12:16:50.156217 25006 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:33569
I20260809 12:16:50.156250 25173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:33569 every 8 connection(s)
I20260809 12:16:50.157508 25006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:16:50.167052 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 25006
I20260809 12:16:50.167261 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:16:50.167847 20191 external_mini_cluster.cc:1428] Setting key 3763306042dbee446115eb754912e308
Aug 09 12:16:50 dist-test-slave-39l8 krb5kdc[24482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277810, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:50.177827 25002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:50.160144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:51921 (local address 127.19.183.254:41455)
0809 12:16:50.160322 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:50.160327 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:50.161551 (+ 1224us) server_negotiation.cc:407] Connection header received
0809 12:16:50.163502 (+ 1951us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:50.163508 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:50.163610 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:50.163749 (+ 139us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:50.165263 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:50.166155 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:50.167987 (+ 1832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:50.168434 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:50.168585 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:50.172540 (+ 3955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:50.172545 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:50.172548 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:50.172581 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:50.175077 (+ 2496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:50.175788 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:50.175792 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:50.175794 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:50.175912 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:50.176278 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:50.176281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:50.176283 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:50.176536 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:50.176661 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:50.177406 (+ 745us) server_negotiation.cc:299] Negotiation successful
0809 12:16:50.177578 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 12:16:50.178952 25178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:50.160580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:41455 (local address 127.19.183.195:51921)
0809 12:16:50.161365 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:50.161405 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:50.163208 (+ 1803us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:50.163978 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:50.163987 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:50.164484 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:50.165066 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:50.165082 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:50.166346 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:50.166351 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:50.167822 (+ 1471us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:50.167836 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:50.168717 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:50.168726 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:50.169046 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:50.169798 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:50.169820 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:50.172349 (+ 2529us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:50.175202 (+ 2853us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:50.175209 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:50.175221 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:50.175665 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:50.176002 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:50.176005 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:50.176007 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:50.176179 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:50.176655 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:50.176661 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:50.177024 (+ 363us) client_negotiation.cc:769] Sending connection context
0809 12:16:50.177293 (+ 269us) client_negotiation.cc:240] Negotiation successful
0809 12:16:50.177689 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":271,"threads_started":1}
I20260809 12:16:50.180356 25174 heartbeater.cc:347] Connected to a master server at 127.19.183.254:41455
I20260809 12:16:50.180668 25174 heartbeater.cc:464] Registering TS with master...
I20260809 12:16:50.181398 25174 heartbeater.cc:511] Master 127.19.183.254:41455 requested a full tablet report, sending...
I20260809 12:16:50.183202 24547 ts_manager.cc:194] Registered new tserver with Master: 1265a494f5724aa9bc0bdcb02b772967 (127.19.183.195:33569)
I20260809 12:16:50.184082 24547 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:51921
I20260809 12:16:50.195825 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 12:16:50 dist-test-slave-39l8 krb5kdc[24482](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277810, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 12:16:50 dist-test-slave-39l8 krb5kdc[24482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277810, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:50.256183 25002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:50.242026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53408 (local address 127.19.183.254:41455)
0809 12:16:50.242221 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:50.242226 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:50.242861 (+ 635us) server_negotiation.cc:407] Connection header received
0809 12:16:50.242965 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:50.242970 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:50.243046 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:50.243142 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:50.244458 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:50.245309 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:50.246538 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:50.247053 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:50.247153 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:50.250955 (+ 3802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:50.250960 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:50.250963 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:50.250999 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:50.253673 (+ 2674us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:50.254285 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:50.254289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:50.254291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:50.254422 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:50.254818 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:50.254821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:50.254823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:50.255144 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:50.255351 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:50.255641 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 12:16:50.255806 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 12:16:50.256870 24547 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:53408
I20260809 12:16:50.259336 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 24628
Aug 09 12:16:50 dist-test-slave-39l8 krb5kdc[24482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277810, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.19.183.193@KRBTEST.COM
I20260809 12:16:50.274801 25192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:50.260238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58568 (local address 127.19.183.193:33853)
0809 12:16:50.260843 (+ 605us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:50.260850 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:50.260872 (+ 22us) server_negotiation.cc:407] Connection header received
0809 12:16:50.260928 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:50.260933 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:50.261129 (+ 196us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:50.261254 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:50.262346 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:50.263442 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:50.264578 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:50.265122 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:50.265216 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:50.269149 (+ 3933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:50.269161 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:50.269174 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:50.269205 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:50.272424 (+ 3219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:50.273017 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:50.273024 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:50.273027 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:50.273149 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:50.273475 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:50.273478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:50.273480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:50.273827 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:50.274051 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:50.274241 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 12:16:50.274441 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":454,"thread_start_us":309,"threads_started":1}
W20260809 12:16:50.276085 24800 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:58568
W20260809 12:16:50.277182 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 24628: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:16:50.277405 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 24628
I20260809 12:16:50.317997 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 24813
Aug 09 12:16:50 dist-test-slave-39l8 krb5kdc[24482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277810, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.19.183.194@KRBTEST.COM
I20260809 12:16:50.333930 25193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:50.318907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37402 (local address 127.19.183.194:43777)
0809 12:16:50.319463 (+ 556us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:50.319471 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:50.319499 (+ 28us) server_negotiation.cc:407] Connection header received
0809 12:16:50.319565 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:50.319571 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:50.319784 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:50.319918 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:50.321190 (+ 1272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:50.322289 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:50.323615 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:50.324179 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:50.324295 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:50.328303 (+ 4008us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:50.328314 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:50.328328 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:50.328366 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:50.331533 (+ 3167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:50.332162 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:50.332169 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:50.332171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:50.332298 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:50.332626 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:50.332630 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:50.332632 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:50.332959 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:50.333200 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:50.333446 (+ 246us) server_negotiation.cc:299] Negotiation successful
0809 12:16:50.333644 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":297,"threads_started":1}
W20260809 12:16:50.335412 24994 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37402
W20260809 12:16:50.336513 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 24813: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:16:50.336674 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 24813
I20260809 12:16:50.367923 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 25006
Aug 09 12:16:50 dist-test-slave-39l8 krb5kdc[24482](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277810, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.19.183.195@KRBTEST.COM
I20260809 12:16:50.384002 25194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:50.368968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38742 (local address 127.19.183.195:33569)
0809 12:16:50.369586 (+ 618us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:50.369595 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:50.369625 (+ 30us) server_negotiation.cc:407] Connection header received
0809 12:16:50.369689 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:50.369695 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:50.369904 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:50.370025 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:50.371427 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:50.372487 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:50.373649 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:50.374178 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:50.374276 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:50.378493 (+ 4217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:50.378505 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:50.378520 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:50.378553 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:50.381460 (+ 2907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:50.382069 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:50.382075 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:50.382078 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:50.382200 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:50.382494 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:50.382497 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:50.382499 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:50.382895 (+ 396us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:50.383155 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:50.383375 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 12:16:50.383574 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":314,"threads_started":1}
W20260809 12:16:50.385371 25166 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:38742
W20260809 12:16:50.386518 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 25006: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:16:50.386838 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 25006
I20260809 12:16:50.423069 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 24501
I20260809 12:16:50.424309 24587 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:16:50.528954 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 24501
2026-08-09T12:16:50Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (3993 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:16:50 dist-test-slave-39l8 krb5kdc[25199](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:16:50 dist-test-slave-39l8 krb5kdc[25199](info): set up 2 sockets
Aug 09 12:16:50 dist-test-slave-39l8 krb5kdc[25199](info): commencing operation
krb5kdc: starting...
W20260809 12:16:51.807770 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.228s user 0.000s sys 0.028s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:16:51 dist-test-slave-39l8 krb5kdc[25199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:16:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:16:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:16:52.032099 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:35207
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:35049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:35207
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:52.370383 25218 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:52.370663 25218 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:52.370828 25218 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:52.377784 25218 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:16:52.377899 25218 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:52.377947 25218 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:52.377998 25218 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:16:52.378043 25218 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:16:52.387449 25218 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:35049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:35207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:35207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:52.388716 25218 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:52.390003 25218 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:52.401993 25225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:52.402171 25226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:52.402253 25228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:52.403044 25218 server_base.cc:1034] running on GCE node
I20260809 12:16:52.403834 25218 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:52.405135 25218 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:52.406348 25218 hybrid_clock.cc:648] HybridClock initialized: now 1786277812406297 us; error 37 us; skew 500 ppm
I20260809 12:16:52.406766 25218 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:52 dist-test-slave-39l8 krb5kdc[25199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277812, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:52.414315 25218 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:16:52.415997 25218 webserver.cc:460] Webserver started at http://127.19.183.254:46743/ using document root <none> and password file <none>
I20260809 12:16:52.416734 25218 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:52.416882 25218 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:52.417207 25218 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:52.420375 25218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "590ec75ad3b7446d9446e5759c2a7029"
format_stamp: "Formatted at 2026-08-09 12:16:52 on dist-test-slave-39l8"
server_key: "5fea670114eb64cdb77c4d10443d4b10"
server_key_iv: "b0f061a673839b1fee885d9b9842e893"
server_key_version: "encryptionkey@0"
I20260809 12:16:52.421273 25218 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "590ec75ad3b7446d9446e5759c2a7029"
format_stamp: "Formatted at 2026-08-09 12:16:52 on dist-test-slave-39l8"
server_key: "5fea670114eb64cdb77c4d10443d4b10"
server_key_iv: "b0f061a673839b1fee885d9b9842e893"
server_key_version: "encryptionkey@0"
I20260809 12:16:52.426610 25218 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 12:16:52.430068 25238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:52.431384 25218 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 12:16:52.431577 25218 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "590ec75ad3b7446d9446e5759c2a7029"
format_stamp: "Formatted at 2026-08-09 12:16:52 on dist-test-slave-39l8"
server_key: "5fea670114eb64cdb77c4d10443d4b10"
server_key_iv: "b0f061a673839b1fee885d9b9842e893"
server_key_version: "encryptionkey@0"
I20260809 12:16:52.431779 25218 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:52.447991 25218 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:52.452255 25218 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:52.452553 25218 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:52.472152 25218 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:35207
I20260809 12:16:52.472186 25307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:35207 every 8 connection(s)
I20260809 12:16:52.473480 25218 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:16:52.474234 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 25218
I20260809 12:16:52.474472 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:16:52.475245 20191 external_mini_cluster.cc:1428] Setting key 75c04d2b3ec14ee79d56673a6e17613a
I20260809 12:16:52.478128 25309 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 12:16:52 dist-test-slave-39l8 krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277811, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:52.488641 25309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 590ec75ad3b7446d9446e5759c2a7029: Bootstrap starting.
I20260809 12:16:52.492079 25309 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 590ec75ad3b7446d9446e5759c2a7029: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:52.493148 25309 log.cc:824] T 00000000000000000000000000000000 P 590ec75ad3b7446d9446e5759c2a7029: Log is configured to *not* fsync() on all Append() calls
I20260809 12:16:52.496682 25309 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 590ec75ad3b7446d9446e5759c2a7029: No bootstrap required, opened a new log
I20260809 12:16:52.497412 25313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:52.477228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36068 (local address 127.19.183.254:35207)
0809 12:16:52.478073 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:52.478085 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:52.478129 (+ 44us) server_negotiation.cc:407] Connection header received
0809 12:16:52.479873 (+ 1744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:52.479893 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:52.480233 (+ 340us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:52.480613 (+ 380us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:52.482102 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:52.483503 (+ 1401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:52.484721 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:52.485381 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:52.485596 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:52.489548 (+ 3952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:52.489555 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:52.489568 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:52.489606 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:52.492743 (+ 3137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:52.493433 (+ 690us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:52.493441 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:52.493448 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:52.493681 (+ 233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:52.494247 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:52.494251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:52.494254 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:52.494953 (+ 699us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:52.495270 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:52.495647 (+ 377us) server_negotiation.cc:299] Negotiation successful
0809 12:16:52.496078 (+ 431us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":584,"thread_start_us":365,"threads_started":1}
I20260809 12:16:52.505235 25309 raft_consensus.cc:348] T 00000000000000000000000000000000 P 590ec75ad3b7446d9446e5759c2a7029 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590ec75ad3b7446d9446e5759c2a7029" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35207 } }
I20260809 12:16:52.505555 25309 raft_consensus.cc:374] T 00000000000000000000000000000000 P 590ec75ad3b7446d9446e5759c2a7029 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:52.505645 25309 raft_consensus.cc:729] T 00000000000000000000000000000000 P 590ec75ad3b7446d9446e5759c2a7029 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 590ec75ad3b7446d9446e5759c2a7029, State: Initialized, Role: FOLLOWER
I20260809 12:16:52.506248 25309 consensus_queue.cc:260] T 00000000000000000000000000000000 P 590ec75ad3b7446d9446e5759c2a7029 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590ec75ad3b7446d9446e5759c2a7029" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35207 } }
I20260809 12:16:52.506520 25309 raft_consensus.cc:388] T 00000000000000000000000000000000 P 590ec75ad3b7446d9446e5759c2a7029 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:16:52.506688 25309 raft_consensus.cc:482] T 00000000000000000000000000000000 P 590ec75ad3b7446d9446e5759c2a7029 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:16:52.506937 25309 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 590ec75ad3b7446d9446e5759c2a7029 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:52.509023 25309 raft_consensus.cc:504] T 00000000000000000000000000000000 P 590ec75ad3b7446d9446e5759c2a7029 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590ec75ad3b7446d9446e5759c2a7029" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35207 } }
I20260809 12:16:52.509461 25309 leader_election.cc:302] T 00000000000000000000000000000000 P 590ec75ad3b7446d9446e5759c2a7029 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 590ec75ad3b7446d9446e5759c2a7029; no voters:
I20260809 12:16:52.510104 25309 leader_election.cc:288] T 00000000000000000000000000000000 P 590ec75ad3b7446d9446e5759c2a7029 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:16:52.510238 25316 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 590ec75ad3b7446d9446e5759c2a7029 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:16:52.511023 25316 raft_consensus.cc:686] T 00000000000000000000000000000000 P 590ec75ad3b7446d9446e5759c2a7029 [term 1 LEADER]: Becoming Leader. State: Replica: 590ec75ad3b7446d9446e5759c2a7029, State: Running, Role: LEADER
I20260809 12:16:52.511582 25316 consensus_queue.cc:237] T 00000000000000000000000000000000 P 590ec75ad3b7446d9446e5759c2a7029 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590ec75ad3b7446d9446e5759c2a7029" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35207 } }
I20260809 12:16:52.512295 25309 sys_catalog.cc:564] T 00000000000000000000000000000000 P 590ec75ad3b7446d9446e5759c2a7029 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:16:52.514564 25318 sys_catalog.cc:455] T 00000000000000000000000000000000 P 590ec75ad3b7446d9446e5759c2a7029 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 590ec75ad3b7446d9446e5759c2a7029. Latest consensus state: current_term: 1 leader_uuid: "590ec75ad3b7446d9446e5759c2a7029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590ec75ad3b7446d9446e5759c2a7029" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35207 } } }
I20260809 12:16:52.515126 25318 sys_catalog.cc:458] T 00000000000000000000000000000000 P 590ec75ad3b7446d9446e5759c2a7029 [sys.catalog]: This master's current role is: LEADER
I20260809 12:16:52.514806 25317 sys_catalog.cc:455] T 00000000000000000000000000000000 P 590ec75ad3b7446d9446e5759c2a7029 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "590ec75ad3b7446d9446e5759c2a7029" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "590ec75ad3b7446d9446e5759c2a7029" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35207 } } }
I20260809 12:16:52.515427 25317 sys_catalog.cc:458] T 00000000000000000000000000000000 P 590ec75ad3b7446d9446e5759c2a7029 [sys.catalog]: This master's current role is: LEADER
I20260809 12:16:52.517936 25327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:16:52.522508 25327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:16:52.531841 25327 catalog_manager.cc:1347] Generated new cluster ID: 959c654506b046bb85def8fb08062d38
I20260809 12:16:52.531962 25327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:16:52.549261 25327 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:16:52.550655 25327 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:16:52.558089 25327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 590ec75ad3b7446d9446e5759c2a7029: Generated new TSK 0
I20260809 12:16:52.558966 25327 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
I20260809 12:16:52.624224 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35207
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:35049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:52.985873 25352 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:52.986148 25352 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:52.986318 25352 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:52.993642 25352 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:52.993752 25352 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:52.993947 25352 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:16:53.004103 25352 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:35049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:53.005524 25352 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:53.007010 25352 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:53.021258 25357 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:53.021296 25361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:53.021203 25359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:53.022913 25352 server_base.cc:1034] running on GCE node
I20260809 12:16:53.023548 25352 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:53.024565 25352 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:53.025794 25352 hybrid_clock.cc:648] HybridClock initialized: now 1786277813025749 us; error 36 us; skew 500 ppm
I20260809 12:16:53.026104 25352 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:53 dist-test-slave-39l8 krb5kdc[25199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277813, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:53.029479 25352 init.cc:377] Logged in from keytab as kudu/127.19.183.193@KRBTEST.COM (short username kudu)
I20260809 12:16:53.031013 25352 webserver.cc:460] Webserver started at http://127.19.183.193:46393/ using document root <none> and password file <none>
I20260809 12:16:53.031808 25352 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:53.031940 25352 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:53.032298 25352 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:53.035442 25352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "7982656508224ae8af978cdc5464ba62"
format_stamp: "Formatted at 2026-08-09 12:16:53 on dist-test-slave-39l8"
server_key: "67bf70abfe0ae40e665c613aa519a6d7"
server_key_iv: "42af2892987926fb25fddb6ac406c8f0"
server_key_version: "encryptionkey@0"
I20260809 12:16:53.036319 25352 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "7982656508224ae8af978cdc5464ba62"
format_stamp: "Formatted at 2026-08-09 12:16:53 on dist-test-slave-39l8"
server_key: "67bf70abfe0ae40e665c613aa519a6d7"
server_key_iv: "42af2892987926fb25fddb6ac406c8f0"
server_key_version: "encryptionkey@0"
I20260809 12:16:53.041718 25352 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 12:16:53.045377 25374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:53.046730 25352 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 12:16:53.046911 25352 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "7982656508224ae8af978cdc5464ba62"
format_stamp: "Formatted at 2026-08-09 12:16:53 on dist-test-slave-39l8"
server_key: "67bf70abfe0ae40e665c613aa519a6d7"
server_key_iv: "42af2892987926fb25fddb6ac406c8f0"
server_key_version: "encryptionkey@0"
I20260809 12:16:53.047150 25352 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:53.077893 25352 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:53.082172 25352 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:53.082489 25352 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:53.083503 25352 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:16:53.084833 25352 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:53.084909 25352 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:53.085001 25352 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:53.085047 25352 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:53.120296 25352 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:46681
I20260809 12:16:53.120349 25534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:46681 every 8 connection(s)
I20260809 12:16:53.121688 25352 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:16:53.123438 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 25352
I20260809 12:16:53.123677 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:16:53.124536 20191 external_mini_cluster.cc:1428] Setting key 4d955a81d420ce244c764b108f338cfd
Aug 09 12:16:53 dist-test-slave-39l8 krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277813, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:53.144537 25537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:53.124702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:54945 (local address 127.19.183.254:35207)
0809 12:16:53.125243 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:53.125249 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:53.126590 (+ 1341us) server_negotiation.cc:407] Connection header received
0809 12:16:53.128606 (+ 2016us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:53.128612 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:53.128704 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:53.128821 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:53.130337 (+ 1516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:53.131139 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:53.132736 (+ 1597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:53.133142 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:53.133295 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:53.137166 (+ 3871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:53.137174 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:53.137182 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:53.137228 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:53.141063 (+ 3835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:53.141785 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:53.141794 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:53.141800 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:53.141981 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:53.142429 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:53.142437 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:53.142443 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:53.142890 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:53.143051 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:53.143788 (+ 737us) server_negotiation.cc:299] Negotiation successful
0809 12:16:53.144060 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":259,"threads_started":1}
I20260809 12:16:53.145177 25538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:53.125409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35207 (local address 127.19.183.193:54945)
0809 12:16:53.126398 (+ 989us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:53.126462 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:53.128340 (+ 1878us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:53.129080 (+ 740us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:53.129088 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:53.129596 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:53.130159 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:53.130177 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:53.131307 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:53.131320 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:53.132581 (+ 1261us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:53.132593 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:53.133434 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:53.133443 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:53.133681 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:53.134402 (+ 721us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:53.134430 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:53.136929 (+ 2499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:53.141211 (+ 4282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:53.141218 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:53.141232 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:53.141655 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:53.142127 (+ 472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:53.142130 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:53.142132 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:53.142314 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:53.143024 (+ 710us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:53.143030 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:53.143371 (+ 341us) client_negotiation.cc:769] Sending connection context
0809 12:16:53.143607 (+ 236us) client_negotiation.cc:240] Negotiation successful
0809 12:16:53.143995 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":380,"threads_started":1}
I20260809 12:16:53.146938 25535 heartbeater.cc:347] Connected to a master server at 127.19.183.254:35207
I20260809 12:16:53.147294 25535 heartbeater.cc:464] Registering TS with master...
I20260809 12:16:53.148278 25535 heartbeater.cc:511] Master 127.19.183.254:35207 requested a full tablet report, sending...
I20260809 12:16:53.150786 25264 ts_manager.cc:194] Registered new tserver with Master: 7982656508224ae8af978cdc5464ba62 (127.19.183.193:46681)
WARNING: no policy specified for kudu/127.19.183.194@KRBTEST.COM; defaulting to no policy
I20260809 12:16:53.152477 25264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:54945
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
I20260809 12:16:53.179702 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35207
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:35049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:53.519475 25544 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:53.519768 25544 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:53.519863 25544 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:53.527069 25544 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:53.527227 25544 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:53.527402 25544 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:16:53.537736 25544 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:35049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:53.539047 25544 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:53.540329 25544 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:53.552973 25551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:53.553480 25552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:53.553531 25555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:53.554462 25544 server_base.cc:1034] running on GCE node
I20260809 12:16:53.555238 25544 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:53.556242 25544 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:53.557497 25544 hybrid_clock.cc:648] HybridClock initialized: now 1786277813557438 us; error 60 us; skew 500 ppm
I20260809 12:16:53.557832 25544 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:53 dist-test-slave-39l8 krb5kdc[25199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277813, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:53.564245 25544 init.cc:377] Logged in from keytab as kudu/127.19.183.194@KRBTEST.COM (short username kudu)
I20260809 12:16:53.565759 25544 webserver.cc:460] Webserver started at http://127.19.183.194:36643/ using document root <none> and password file <none>
I20260809 12:16:53.566476 25544 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:53.566624 25544 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:53.566965 25544 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:53.569816 25544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "fd51a4abaf534536bbb259c4813926c5"
format_stamp: "Formatted at 2026-08-09 12:16:53 on dist-test-slave-39l8"
server_key: "2ee2d60b40f32fd2aa4e223411e2692c"
server_key_iv: "52b7c571ec4f82b6c9a3874766ff42c8"
server_key_version: "encryptionkey@0"
I20260809 12:16:53.570645 25544 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "fd51a4abaf534536bbb259c4813926c5"
format_stamp: "Formatted at 2026-08-09 12:16:53 on dist-test-slave-39l8"
server_key: "2ee2d60b40f32fd2aa4e223411e2692c"
server_key_iv: "52b7c571ec4f82b6c9a3874766ff42c8"
server_key_version: "encryptionkey@0"
I20260809 12:16:53.575884 25544 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260809 12:16:53.579263 25563 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:53.580590 25544 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 12:16:53.580785 25544 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "fd51a4abaf534536bbb259c4813926c5"
format_stamp: "Formatted at 2026-08-09 12:16:53 on dist-test-slave-39l8"
server_key: "2ee2d60b40f32fd2aa4e223411e2692c"
server_key_iv: "52b7c571ec4f82b6c9a3874766ff42c8"
server_key_version: "encryptionkey@0"
I20260809 12:16:53.581036 25544 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:53.599658 25544 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:53.603613 25544 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:53.603919 25544 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:53.604897 25544 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:16:53.606315 25544 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:53.606421 25544 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:53.606576 25544 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:53.606654 25544 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:53.646088 25544 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:33883
I20260809 12:16:53.646196 25721 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:33883 every 8 connection(s)
I20260809 12:16:53.647797 25544 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:16:53.652359 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 25544
I20260809 12:16:53.652577 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:16:53.653163 20191 external_mini_cluster.cc:1428] Setting key 04c8fc216ad905f88064081e3bc84306
Aug 09 12:16:53 dist-test-slave-39l8 krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277813, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:53.678833 25725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:53.652182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:48471 (local address 127.19.183.254:35207)
0809 12:16:53.652724 (+ 542us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:53.652732 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:53.654372 (+ 1640us) server_negotiation.cc:407] Connection header received
0809 12:16:53.657243 (+ 2871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:53.657249 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:53.657331 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:53.657502 (+ 171us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:53.659731 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:53.660738 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:53.662347 (+ 1609us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:53.662892 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:53.663066 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:53.670032 (+ 6966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:53.670041 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:53.670046 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:53.670109 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:53.673329 (+ 3220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:53.675578 (+ 2249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:53.675585 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:53.675591 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:53.675748 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:53.676330 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:53.676338 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:53.676343 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:53.676762 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:53.676926 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:53.677994 (+ 1068us) server_negotiation.cc:299] Negotiation successful
0809 12:16:53.678264 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":242,"threads_started":1}
I20260809 12:16:53.680107 25726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:53.652791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35207 (local address 127.19.183.194:48471)
0809 12:16:53.654201 (+ 1410us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:53.654253 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:53.656891 (+ 2638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:53.657767 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:53.657779 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:53.658689 (+ 910us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:53.659530 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:53.659553 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:53.660940 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:53.660948 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:53.662192 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:53.662205 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:53.663450 (+ 1245us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:53.663460 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:53.663852 (+ 392us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:53.666193 (+ 2341us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:53.666234 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:53.669689 (+ 3455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:53.674700 (+ 5011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:53.674761 (+ 61us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:53.674786 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:53.675429 (+ 643us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:53.675910 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:53.675916 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:53.675922 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:53.676162 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:53.676934 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:53.676945 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:53.677471 (+ 526us) client_negotiation.cc:769] Sending connection context
0809 12:16:53.677824 (+ 353us) client_negotiation.cc:240] Negotiation successful
0809 12:16:53.678450 (+ 626us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1076,"thread_start_us":3033,"threads_started":1}
I20260809 12:16:53.682449 25722 heartbeater.cc:347] Connected to a master server at 127.19.183.254:35207
I20260809 12:16:53.682842 25722 heartbeater.cc:464] Registering TS with master...
I20260809 12:16:53.683939 25722 heartbeater.cc:511] Master 127.19.183.254:35207 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.19.183.195@KRBTEST.COM; defaulting to no policy
I20260809 12:16:53.686126 25264 ts_manager.cc:194] Registered new tserver with Master: fd51a4abaf534536bbb259c4813926c5 (127.19.183.194:33883)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.195@KRBTEST.COM" created.
I20260809 12:16:53.687709 25264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:48471
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
I20260809 12:16:53.718268 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35207
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:35049
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:54.054108 25730 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:54.054375 25730 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:54.054466 25730 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:54.061722 25730 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:54.061833 25730 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:54.061942 25730 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:16:54.071857 25730 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:35049
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35207
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:54.073199 25730 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:54.074489 25730 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:54.088155 25737 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:54.088202 25738 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:54.088164 25743 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:54.089159 25730 server_base.cc:1034] running on GCE node
I20260809 12:16:54.089795 25730 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:54.090834 25730 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:54.092096 25730 hybrid_clock.cc:648] HybridClock initialized: now 1786277814092045 us; error 55 us; skew 500 ppm
I20260809 12:16:54.092425 25730 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:54 dist-test-slave-39l8 krb5kdc[25199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277814, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:54.095613 25730 init.cc:377] Logged in from keytab as kudu/127.19.183.195@KRBTEST.COM (short username kudu)
I20260809 12:16:54.097134 25730 webserver.cc:460] Webserver started at http://127.19.183.195:41537/ using document root <none> and password file <none>
I20260809 12:16:54.097846 25730 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:54.097992 25730 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:54.098326 25730 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:54.101204 25730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "fb1f307d679b4eb19d974421f109694e"
format_stamp: "Formatted at 2026-08-09 12:16:54 on dist-test-slave-39l8"
server_key: "731729a8291f43367333e7d0e7f1ade0"
server_key_iv: "b712e39d6cd5e7427d756d74934be49a"
server_key_version: "encryptionkey@0"
I20260809 12:16:54.101959 25730 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "fb1f307d679b4eb19d974421f109694e"
format_stamp: "Formatted at 2026-08-09 12:16:54 on dist-test-slave-39l8"
server_key: "731729a8291f43367333e7d0e7f1ade0"
server_key_iv: "b712e39d6cd5e7427d756d74934be49a"
server_key_version: "encryptionkey@0"
I20260809 12:16:54.107002 25730 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 12:16:54.110351 25752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:54.111709 25730 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:16:54.111887 25730 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "fb1f307d679b4eb19d974421f109694e"
format_stamp: "Formatted at 2026-08-09 12:16:54 on dist-test-slave-39l8"
server_key: "731729a8291f43367333e7d0e7f1ade0"
server_key_iv: "b712e39d6cd5e7427d756d74934be49a"
server_key_version: "encryptionkey@0"
I20260809 12:16:54.112160 25730 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:54.155640 25535 heartbeater.cc:503] Master 127.19.183.254:35207 was elected leader, sending a full tablet report...
I20260809 12:16:54.181329 25730 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:54.185487 25730 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:54.185766 25730 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:54.186873 25730 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:16:54.188235 25730 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:54.188311 25730 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:54.188393 25730 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:54.188438 25730 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:54.222116 25730 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:41977
I20260809 12:16:54.222213 25923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:41977 every 8 connection(s)
I20260809 12:16:54.223472 25730 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:16:54.231946 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 25730
I20260809 12:16:54.232227 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:16:54.232690 20191 external_mini_cluster.cc:1428] Setting key 593d03820335691c5919cdfacddb87ca
Aug 09 12:16:54 dist-test-slave-39l8 krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277814, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:54.243129 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:54.225893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:39461 (local address 127.19.183.254:35207)
0809 12:16:54.226362 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:54.226368 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:54.227463 (+ 1095us) server_negotiation.cc:407] Connection header received
0809 12:16:54.229295 (+ 1832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:54.229300 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:54.229382 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:54.229514 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:54.230951 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:54.231886 (+ 935us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:54.233747 (+ 1861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:54.234320 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:54.234461 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:54.237816 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:54.237821 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:54.237824 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:54.237860 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:54.240581 (+ 2721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:54.241223 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:54.241226 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:54.241228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:54.241353 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:54.241712 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:54.241715 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:54.241717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:54.241941 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:54.242050 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:54.242644 (+ 594us) server_negotiation.cc:299] Negotiation successful
0809 12:16:54.242858 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":209,"threads_started":1}
I20260809 12:16:54.243999 25928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:54.226488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35207 (local address 127.19.183.195:39461)
0809 12:16:54.227211 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:54.227248 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:54.229017 (+ 1769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:54.229723 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:54.229730 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:54.230205 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:54.230807 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:54.230826 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:54.232048 (+ 1222us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:54.232052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:54.233597 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:54.233610 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:54.234560 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:54.234568 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:54.234864 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:54.235502 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:54.235523 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:54.237637 (+ 2114us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:54.240707 (+ 3070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:54.240713 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:54.240724 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:54.241116 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:54.241439 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:54.241442 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:54.241444 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:54.241600 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:54.242042 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:54.242047 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:54.242299 (+ 252us) client_negotiation.cc:769] Sending connection context
0809 12:16:54.242542 (+ 243us) client_negotiation.cc:240] Negotiation successful
0809 12:16:54.242965 (+ 423us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":318,"threads_started":1}
I20260809 12:16:54.245319 25925 heartbeater.cc:347] Connected to a master server at 127.19.183.254:35207
I20260809 12:16:54.245573 25925 heartbeater.cc:464] Registering TS with master...
I20260809 12:16:54.246209 25925 heartbeater.cc:511] Master 127.19.183.254:35207 requested a full tablet report, sending...
I20260809 12:16:54.247907 25264 ts_manager.cc:194] Registered new tserver with Master: fb1f307d679b4eb19d974421f109694e (127.19.183.195:41977)
I20260809 12:16:54.248692 25264 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:39461
I20260809 12:16:54.260339 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 12:16:54 dist-test-slave-39l8 krb5kdc[25199](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 12:16:54 dist-test-slave-39l8 krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.193@KRBTEST.COM
I20260809 12:16:54.316028 25934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:54.301434 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53414 (local address 127.19.183.193:46681)
0809 12:16:54.302058 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:54.302065 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:54.302099 (+ 34us) server_negotiation.cc:407] Connection header received
0809 12:16:54.302163 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:54.302169 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:54.302380 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:54.302525 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:54.303952 (+ 1427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:54.304939 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:54.306083 (+ 1144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:54.306613 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:54.306752 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:54.310368 (+ 3616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:54.310379 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:54.310392 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:54.310422 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:54.313390 (+ 2968us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:54.314051 (+ 661us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:54.314058 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:54.314060 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:54.314208 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:54.314527 (+ 319us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:54.314530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:54.314532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:54.314895 (+ 363us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:54.315131 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:54.315350 (+ 219us) server_negotiation.cc:299] Negotiation successful
0809 12:16:54.315653 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":345,"threads_started":1}
Aug 09 12:16:54 dist-test-slave-39l8 krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:54.333462 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:54.320954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36070 (local address 127.19.183.254:35207)
0809 12:16:54.321199 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:54.321206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:54.321295 (+ 89us) server_negotiation.cc:407] Connection header received
0809 12:16:54.321503 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:54.321508 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:54.321572 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:54.321684 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:54.322964 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:54.323686 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:54.324801 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:54.325212 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:54.325344 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:54.328887 (+ 3543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:54.328891 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:54.328895 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:54.328926 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:54.331294 (+ 2368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:54.331906 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:54.331909 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:54.331911 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:54.332057 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:54.332362 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:54.332365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:54.332367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:54.332654 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:54.332837 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:54.333068 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 12:16:54.333246 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
W20260809 12:16:54.334079 25264 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:36070
I20260809 12:16:54.335131 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 25352
I20260809 12:16:54.346318 25934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:54.335962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53422 (local address 127.19.183.193:46681)
0809 12:16:54.336118 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:54.336122 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:54.336417 (+ 295us) server_negotiation.cc:407] Connection header received
0809 12:16:54.336539 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:54.336542 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:54.336599 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:54.336670 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:54.337673 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:54.338330 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:54.339508 (+ 1178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:54.339886 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:54.339979 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:54.341853 (+ 1874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:54.341857 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:54.341860 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:54.341888 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:54.344333 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:54.344877 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:54.344880 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:54.344882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:54.344987 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:54.345274 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:54.345277 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:54.345279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:54.345513 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:54.345695 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:54.345949 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 12:16:54.346112 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 12:16:54.346887 25528 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:16:54.428966 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 25352
I20260809 12:16:54.453341 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 25544
Aug 09 12:16:54 dist-test-slave-39l8 krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.194@KRBTEST.COM
I20260809 12:16:54.469964 25940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:54.454389 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50734 (local address 127.19.183.194:33883)
0809 12:16:54.455018 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:54.455025 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:54.455052 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:16:54.455118 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:54.455124 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:54.455330 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:54.455461 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:54.456607 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:54.457791 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:54.459191 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:54.460076 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:54.460220 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:54.464414 (+ 4194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:54.464424 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:54.464438 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:54.464473 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:54.467562 (+ 3089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:54.468160 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:54.468167 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:54.468169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:54.468292 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:54.468605 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:54.468609 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:54.468611 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:54.468928 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:54.469165 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:54.469378 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 12:16:54.469570 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":307,"threads_started":1}
I20260809 12:16:54.471308 25715 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:16:54.553916 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 25544
I20260809 12:16:54.580233 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 25730
Aug 09 12:16:54 dist-test-slave-39l8 krb5kdc[25199](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277814, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.195@KRBTEST.COM
I20260809 12:16:54.596503 25943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:54.581256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49066 (local address 127.19.183.195:41977)
0809 12:16:54.581880 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:54.581887 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:54.581916 (+ 29us) server_negotiation.cc:407] Connection header received
0809 12:16:54.581978 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:54.581986 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:54.582229 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:54.582402 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:54.583798 (+ 1396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:54.584816 (+ 1018us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:54.586078 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:54.586620 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:54.586787 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:54.591116 (+ 4329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:54.591128 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:54.591141 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:54.591178 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:54.593702 (+ 2524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:54.594352 (+ 650us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:54.594360 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:54.594362 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:54.594479 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:54.594898 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:54.594903 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:54.594908 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:54.595244 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:54.595568 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:54.595870 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 12:16:54.596151 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":302,"threads_started":1}
I20260809 12:16:54.597755 25918 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:16:54.680716 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 25730
I20260809 12:16:54.707229 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 25218
I20260809 12:16:54.708633 25301 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:16:54.812857 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 25218
2026-08-09T12:16:54Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (4284 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:16:54 dist-test-slave-39l8 krb5kdc[25950](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:16:54 dist-test-slave-39l8 krb5kdc[25950](info): set up 2 sockets
Aug 09 12:16:54 dist-test-slave-39l8 krb5kdc[25950](info): commencing operation
krb5kdc: starting...
W20260809 12:16:56.090250 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.225s user 0.001s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:16:56 dist-test-slave-39l8 krb5kdc[25950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:16:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:16:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:16:56.308142 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:40023
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:40023
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:56.629447 25975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:56.629719 25975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:56.629895 25975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:56.636736 25975 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:16:56.636826 25975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:56.636920 25975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:56.637017 25975 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:16:56.637096 25975 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:16:56.646287 25975 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:40023
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:40023
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:56.648033 25975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:56.649322 25975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:56.660557 25988 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:56.660633 25985 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:56.660744 25986 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:56.661875 25975 server_base.cc:1034] running on GCE node
I20260809 12:16:56.662535 25975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:56.663898 25975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:56.665122 25975 hybrid_clock.cc:648] HybridClock initialized: now 1786277816665092 us; error 56 us; skew 500 ppm
I20260809 12:16:56.665474 25975 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:56 dist-test-slave-39l8 krb5kdc[25950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277816, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:56.668489 25975 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:16:56.669663 25975 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:56.669793 25975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:56.670116 25975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:56.673044 25975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "81aa21b2439244979b942f89e48ff349"
format_stamp: "Formatted at 2026-08-09 12:16:56 on dist-test-slave-39l8"
server_key: "aacd73c5a4955208b7377269cca148eb"
server_key_iv: "86f1013799c1678bd941093c2e7ecd56"
server_key_version: "encryptionkey@0"
I20260809 12:16:56.673853 25975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "81aa21b2439244979b942f89e48ff349"
format_stamp: "Formatted at 2026-08-09 12:16:56 on dist-test-slave-39l8"
server_key: "aacd73c5a4955208b7377269cca148eb"
server_key_iv: "86f1013799c1678bd941093c2e7ecd56"
server_key_version: "encryptionkey@0"
I20260809 12:16:56.678646 25975 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260809 12:16:56.682044 25994 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:56.683314 25975 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:16:56.683511 25975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "81aa21b2439244979b942f89e48ff349"
format_stamp: "Formatted at 2026-08-09 12:16:56 on dist-test-slave-39l8"
server_key: "aacd73c5a4955208b7377269cca148eb"
server_key_iv: "86f1013799c1678bd941093c2e7ecd56"
server_key_version: "encryptionkey@0"
I20260809 12:16:56.683737 25975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:56.696372 25975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:56.700459 25975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:56.700724 25975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:56.719316 25975 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:40023
I20260809 12:16:56.719336 26059 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:40023 every 8 connection(s)
I20260809 12:16:56.720256 25975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:16:56.723639 26060 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:56.727249 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 25975
I20260809 12:16:56.727483 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:16:56.728374 20191 external_mini_cluster.cc:1428] Setting key 80e759ef8ebf78229d1d5843e68b62c1
I20260809 12:16:56.736946 26060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349: Bootstrap starting.
I20260809 12:16:56.740726 26060 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:56.742055 26060 log.cc:824] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349: Log is configured to *not* fsync() on all Append() calls
Aug 09 12:16:56 dist-test-slave-39l8 krb5kdc[25950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:56.744709 26060 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349: No bootstrap required, opened a new log
I20260809 12:16:56.752195 26064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:56.730433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56370 (local address 127.19.183.254:40023)
0809 12:16:56.731279 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:56.731290 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:56.731331 (+ 41us) server_negotiation.cc:407] Connection header received
0809 12:16:56.732946 (+ 1615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:56.732969 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:56.733347 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:56.733647 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:56.735505 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:56.736716 (+ 1211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:56.738475 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:56.739238 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:56.739381 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:56.744060 (+ 4679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:56.744068 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:56.744081 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:56.744117 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:56.747990 (+ 3873us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:56.748631 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:56.748640 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:56.748648 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:56.748861 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:56.749228 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:56.749234 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:56.749238 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:56.749908 (+ 670us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:56.750202 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:56.750488 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 12:16:56.750932 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":413,"threads_started":1}
I20260809 12:16:56.754582 26060 raft_consensus.cc:348] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81aa21b2439244979b942f89e48ff349" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40023 } }
I20260809 12:16:56.754978 26060 raft_consensus.cc:374] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:56.755053 26060 raft_consensus.cc:729] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 81aa21b2439244979b942f89e48ff349, State: Initialized, Role: FOLLOWER
I20260809 12:16:56.755571 26060 consensus_queue.cc:260] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81aa21b2439244979b942f89e48ff349" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40023 } }
I20260809 12:16:56.755764 26060 raft_consensus.cc:388] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:16:56.755865 26060 raft_consensus.cc:482] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:16:56.756040 26060 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:56.758195 26060 raft_consensus.cc:504] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81aa21b2439244979b942f89e48ff349" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40023 } }
I20260809 12:16:56.758689 26060 leader_election.cc:302] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 81aa21b2439244979b942f89e48ff349; no voters:
I20260809 12:16:56.759322 26060 leader_election.cc:288] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:16:56.759475 26066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:16:56.760187 26066 raft_consensus.cc:686] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349 [term 1 LEADER]: Becoming Leader. State: Replica: 81aa21b2439244979b942f89e48ff349, State: Running, Role: LEADER
I20260809 12:16:56.760764 26066 consensus_queue.cc:237] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81aa21b2439244979b942f89e48ff349" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40023 } }
I20260809 12:16:56.761402 26060 sys_catalog.cc:564] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:16:56.764187 26068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 81aa21b2439244979b942f89e48ff349. Latest consensus state: current_term: 1 leader_uuid: "81aa21b2439244979b942f89e48ff349" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81aa21b2439244979b942f89e48ff349" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40023 } } }
I20260809 12:16:56.764616 26068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349 [sys.catalog]: This master's current role is: LEADER
I20260809 12:16:56.765232 26067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "81aa21b2439244979b942f89e48ff349" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "81aa21b2439244979b942f89e48ff349" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40023 } } }
I20260809 12:16:56.765615 26067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349 [sys.catalog]: This master's current role is: LEADER
I20260809 12:16:56.768121 26074 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:16:56.773989 26074 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:16:56.781973 26074 catalog_manager.cc:1347] Generated new cluster ID: 821982a5049b49bca8924ecc2e0a129a
I20260809 12:16:56.782076 26074 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:16:56.793253 26074 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:16:56.794974 26074 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:16:56.804075 26074 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349: Generated new TSK 0
I20260809 12:16:56.805006 26074 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
I20260809 12:16:56.859436 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40023
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:57.195026 26090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:57.195344 26090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:57.195531 26090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:57.202536 26090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:57.202683 26090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:57.202906 26090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:16:57.212446 26090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:57.213749 26090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:57.215137 26090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:57.228264 26098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:57.228320 26099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:57.228364 26101 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:57.229697 26090 server_base.cc:1034] running on GCE node
I20260809 12:16:57.230298 26090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:57.231290 26090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:57.232586 26090 hybrid_clock.cc:648] HybridClock initialized: now 1786277817232555 us; error 44 us; skew 500 ppm
I20260809 12:16:57.232865 26090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:57 dist-test-slave-39l8 krb5kdc[25950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277817, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:57.236060 26090 init.cc:377] Logged in from keytab as kudu/127.19.183.193@KRBTEST.COM (short username kudu)
I20260809 12:16:57.237159 26090 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:57.237250 26090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:57.237515 26090 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:57.240470 26090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "b83194fddfa346099ceeff7e81b16266"
format_stamp: "Formatted at 2026-08-09 12:16:57 on dist-test-slave-39l8"
server_key: "f22edca8223b3ccaadc71b62471f01ce"
server_key_iv: "1d8df58c356a3cc1fe00e3ec3840142e"
server_key_version: "encryptionkey@0"
I20260809 12:16:57.241257 26090 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "b83194fddfa346099ceeff7e81b16266"
format_stamp: "Formatted at 2026-08-09 12:16:57 on dist-test-slave-39l8"
server_key: "f22edca8223b3ccaadc71b62471f01ce"
server_key_iv: "1d8df58c356a3cc1fe00e3ec3840142e"
server_key_version: "encryptionkey@0"
I20260809 12:16:57.246299 26090 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:16:57.249745 26111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:57.251276 26090 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 12:16:57.251462 26090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "b83194fddfa346099ceeff7e81b16266"
format_stamp: "Formatted at 2026-08-09 12:16:57 on dist-test-slave-39l8"
server_key: "f22edca8223b3ccaadc71b62471f01ce"
server_key_iv: "1d8df58c356a3cc1fe00e3ec3840142e"
server_key_version: "encryptionkey@0"
I20260809 12:16:57.251689 26090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:57.285012 26090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:57.289619 26090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:57.289904 26090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:57.290966 26090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:16:57.292356 26090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:57.292443 26090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:57.292599 26090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:57.292698 26090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:57.326561 26090 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:40735
I20260809 12:16:57.326612 26288 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:40735 every 8 connection(s)
I20260809 12:16:57.327461 26090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:16:57.333482 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 26090
I20260809 12:16:57.333721 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:16:57.334558 20191 external_mini_cluster.cc:1428] Setting key d804f682081116e087ed31486d352be4
Aug 09 12:16:57 dist-test-slave-39l8 krb5kdc[25950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277817, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:57.349454 26294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:57.329882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:55191 (local address 127.19.183.254:40023)
0809 12:16:57.330357 (+ 475us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:57.330365 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:57.331383 (+ 1018us) server_negotiation.cc:407] Connection header received
0809 12:16:57.333200 (+ 1817us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:57.333205 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:57.333287 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:57.333383 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:57.335505 (+ 2122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:57.336561 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:57.339042 (+ 2481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:57.339441 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:57.339537 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:57.343733 (+ 4196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:57.343738 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:57.343742 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:57.343772 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:57.346303 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:57.347158 (+ 855us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:57.347168 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:57.347171 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:57.347288 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:57.347718 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:57.347725 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:57.347733 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:57.347988 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:57.348172 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:57.348857 (+ 685us) server_negotiation.cc:299] Negotiation successful
0809 12:16:57.349077 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":345,"thread_start_us":227,"threads_started":1}
I20260809 12:16:57.350307 26295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:57.330458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:40023 (local address 127.19.183.193:55191)
0809 12:16:57.331208 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:57.331246 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:57.332884 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:57.333677 (+ 793us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:57.333690 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:57.334452 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:57.335307 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:57.335328 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:57.336770 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:57.336775 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:57.338886 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:57.338907 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:57.339685 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:57.339694 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:57.340021 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:57.340755 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:57.340789 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:57.343504 (+ 2715us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:57.346461 (+ 2957us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:57.346469 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:57.346483 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:57.347005 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:57.347404 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:57.347409 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:57.347413 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:57.347591 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:57.348140 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:57.348149 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:57.348470 (+ 321us) client_negotiation.cc:769] Sending connection context
0809 12:16:57.348735 (+ 265us) client_negotiation.cc:240] Negotiation successful
0809 12:16:57.349234 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":525,"thread_start_us":344,"threads_started":1}
I20260809 12:16:57.351826 26289 heartbeater.cc:347] Connected to a master server at 127.19.183.254:40023
I20260809 12:16:57.352197 26289 heartbeater.cc:464] Registering TS with master...
I20260809 12:16:57.353022 26289 heartbeater.cc:511] Master 127.19.183.254:40023 requested a full tablet report, sending...
I20260809 12:16:57.355371 26017 ts_manager.cc:194] Registered new tserver with Master: b83194fddfa346099ceeff7e81b16266 (127.19.183.193:40735)
I20260809 12:16:57.356971 26017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:55191
WARNING: no policy specified for kudu/127.19.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
I20260809 12:16:57.387610 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40023
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:57.728075 26300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:57.728506 26300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:57.728684 26300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:57.742002 26300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:57.742151 26300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:57.742327 26300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:16:57.752856 26300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:57.754146 26300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:57.755556 26300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:57.768533 26308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:57.768780 26310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:57.769133 26314 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:57.770000 26300 server_base.cc:1034] running on GCE node
I20260809 12:16:57.770642 26300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:57.771590 26300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:57.772825 26300 hybrid_clock.cc:648] HybridClock initialized: now 1786277817772784 us; error 51 us; skew 500 ppm
I20260809 12:16:57.773160 26300 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:57 dist-test-slave-39l8 krb5kdc[25950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277817, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:57.780689 26300 init.cc:377] Logged in from keytab as kudu/127.19.183.194@KRBTEST.COM (short username kudu)
I20260809 12:16:57.781994 26300 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:57.782136 26300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:57.782459 26300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:57.785529 26300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "ed350d3b6a66454f9fb325d13b31d374"
format_stamp: "Formatted at 2026-08-09 12:16:57 on dist-test-slave-39l8"
server_key: "290d201b9e44d896119ba2ca1db7c407"
server_key_iv: "d27498b2bb9dcd044aae9c2693df5ddc"
server_key_version: "encryptionkey@0"
I20260809 12:16:57.786358 26300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "ed350d3b6a66454f9fb325d13b31d374"
format_stamp: "Formatted at 2026-08-09 12:16:57 on dist-test-slave-39l8"
server_key: "290d201b9e44d896119ba2ca1db7c407"
server_key_iv: "d27498b2bb9dcd044aae9c2693df5ddc"
server_key_version: "encryptionkey@0"
I20260809 12:16:57.791577 26300 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:16:57.795111 26321 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:57.796420 26300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 12:16:57.796599 26300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "ed350d3b6a66454f9fb325d13b31d374"
format_stamp: "Formatted at 2026-08-09 12:16:57 on dist-test-slave-39l8"
server_key: "290d201b9e44d896119ba2ca1db7c407"
server_key_iv: "d27498b2bb9dcd044aae9c2693df5ddc"
server_key_version: "encryptionkey@0"
I20260809 12:16:57.796825 26300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:57.816220 26300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:57.820571 26300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:57.820847 26300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:57.821831 26300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:16:57.823282 26300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:57.823405 26300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:57.823536 26300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:57.823625 26300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:57.858847 26300 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:44863
I20260809 12:16:57.858932 26493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:44863 every 8 connection(s)
I20260809 12:16:57.859745 26300 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:16:57.870213 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 26300
I20260809 12:16:57.870414 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:16:57.870929 20191 external_mini_cluster.cc:1428] Setting key 03270a31b46ef2bc3bb188e0379dee2d
Aug 09 12:16:57 dist-test-slave-39l8 krb5kdc[25950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277817, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:57.879954 26497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:57.862178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:60575 (local address 127.19.183.254:40023)
0809 12:16:57.862680 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:57.862684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:57.863614 (+ 930us) server_negotiation.cc:407] Connection header received
0809 12:16:57.865436 (+ 1822us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:57.865444 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:57.865531 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:57.865660 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:57.867313 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:57.868056 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:57.869752 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:57.870342 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:57.870466 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:57.874183 (+ 3717us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:57.874188 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:57.874192 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:57.874223 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:57.876785 (+ 2562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:57.877556 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:57.877564 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:57.877572 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:57.877696 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:57.878158 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:57.878162 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:57.878164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:57.878426 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:57.878587 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:57.879334 (+ 747us) server_negotiation.cc:299] Negotiation successful
0809 12:16:57.879574 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":289,"threads_started":1}
I20260809 12:16:57.880900 26498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:57.862628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:40023 (local address 127.19.183.194:60575)
0809 12:16:57.863436 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:57.863474 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:57.865154 (+ 1680us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:57.865897 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:57.865904 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:57.866405 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:57.867102 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:57.867122 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:57.868224 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:57.868234 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:57.869590 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:57.869604 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:57.870565 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:57.870572 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:57.870850 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:57.871562 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:57.871583 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:57.873949 (+ 2366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:57.876941 (+ 2992us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:57.876949 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:57.876965 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:57.877405 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:57.877824 (+ 419us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:57.877829 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:57.877833 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:57.878021 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:57.878572 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:57.878580 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:57.878969 (+ 389us) client_negotiation.cc:769] Sending connection context
0809 12:16:57.879244 (+ 275us) client_negotiation.cc:240] Negotiation successful
0809 12:16:57.879766 (+ 522us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":562,"thread_start_us":320,"threads_started":1}
I20260809 12:16:57.882400 26495 heartbeater.cc:347] Connected to a master server at 127.19.183.254:40023
I20260809 12:16:57.882752 26495 heartbeater.cc:464] Registering TS with master...
I20260809 12:16:57.883405 26495 heartbeater.cc:511] Master 127.19.183.254:40023 requested a full tablet report, sending...
I20260809 12:16:57.884872 26017 ts_manager.cc:194] Registered new tserver with Master: ed350d3b6a66454f9fb325d13b31d374 (127.19.183.194:44863)
I20260809 12:16:57.885661 26017 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:60575
WARNING: no policy specified for kudu/127.19.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
I20260809 12:16:57.923506 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40023
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40019
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:16:58.262991 26501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:58.263330 26501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:58.263450 26501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:58.270778 26501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:58.270876 26501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:58.271039 26501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:16:58.280515 26501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40019
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40023
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:58.281668 26501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:58.282960 26501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:58.295809 26509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:58.295814 26508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:58.295853 26512 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:58.296854 26501 server_base.cc:1034] running on GCE node
I20260809 12:16:58.297436 26501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:58.298336 26501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:58.299536 26501 hybrid_clock.cc:648] HybridClock initialized: now 1786277818299463 us; error 52 us; skew 500 ppm
I20260809 12:16:58.299868 26501 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:16:58 dist-test-slave-39l8 krb5kdc[25950](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277818, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:16:58.303076 26501 init.cc:377] Logged in from keytab as kudu/127.19.183.195@KRBTEST.COM (short username kudu)
I20260809 12:16:58.304183 26501 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:58.304296 26501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:58.304612 26501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:58.307518 26501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "f34c1cc441784a619507b050435e07e7"
format_stamp: "Formatted at 2026-08-09 12:16:58 on dist-test-slave-39l8"
server_key: "a5f251a1085224757c24018833281ebd"
server_key_iv: "250d6fe6a2e247e7411ed0971c44783a"
server_key_version: "encryptionkey@0"
I20260809 12:16:58.308303 26501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "f34c1cc441784a619507b050435e07e7"
format_stamp: "Formatted at 2026-08-09 12:16:58 on dist-test-slave-39l8"
server_key: "a5f251a1085224757c24018833281ebd"
server_key_iv: "250d6fe6a2e247e7411ed0971c44783a"
server_key_version: "encryptionkey@0"
I20260809 12:16:58.313174 26501 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 12:16:58.316383 26519 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:58.317647 26501 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 12:16:58.317821 26501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "f34c1cc441784a619507b050435e07e7"
format_stamp: "Formatted at 2026-08-09 12:16:58 on dist-test-slave-39l8"
server_key: "a5f251a1085224757c24018833281ebd"
server_key_iv: "250d6fe6a2e247e7411ed0971c44783a"
server_key_version: "encryptionkey@0"
I20260809 12:16:58.318078 26501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:58.349012 26501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:58.353367 26501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:58.353667 26501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:58.354673 26501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:16:58.356168 26501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:16:58.356256 26501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:58.356334 26501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:16:58.356401 26501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:58.360466 26289 heartbeater.cc:503] Master 127.19.183.254:40023 was elected leader, sending a full tablet report...
I20260809 12:16:58.391491 26501 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:40153
I20260809 12:16:58.391541 26709 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:40153 every 8 connection(s)
I20260809 12:16:58.392397 26501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:16:58.397779 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 26501
I20260809 12:16:58.397992 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:16:58.398531 20191 external_mini_cluster.cc:1428] Setting key 8fd87b8b22780e5f560e2ba219023497
Aug 09 12:16:58 dist-test-slave-39l8 krb5kdc[25950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277818, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:16:58.412230 26712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:58.394861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:53347 (local address 127.19.183.254:40023)
0809 12:16:58.395315 (+ 454us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:58.395320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:58.396110 (+ 790us) server_negotiation.cc:407] Connection header received
0809 12:16:58.398086 (+ 1976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:58.398093 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:58.398194 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:58.398294 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:58.400219 (+ 1925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.401449 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.403234 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.403662 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.403760 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:58.406964 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:58.406969 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:58.406972 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:58.407004 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:58.409681 (+ 2677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:58.410300 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:58.410304 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:58.410306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:58.410419 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:58.410809 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:58.410813 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:58.410814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:58.411068 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:58.411188 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:58.411786 (+ 598us) server_negotiation.cc:299] Negotiation successful
0809 12:16:58.411949 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":238,"threads_started":1}
I20260809 12:16:58.413218 26713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:58.395284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:40023 (local address 127.19.183.195:53347)
0809 12:16:58.395957 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:58.395992 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:58.397761 (+ 1769us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:58.398584 (+ 823us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:58.398595 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:58.399285 (+ 690us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:16:58.399990 (+ 705us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:58.400041 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.401613 (+ 1572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.401618 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:58.403087 (+ 1469us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:58.403117 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.403846 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.403851 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:58.404014 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:58.404592 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:16:58.404633 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:16:58.406780 (+ 2147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:16:58.409802 (+ 3022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:58.409808 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:58.409817 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:58.410193 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:58.410510 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:16:58.410513 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:16:58.410515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:16:58.410665 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:16:58.411181 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:16:58.411186 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:16:58.411465 (+ 279us) client_negotiation.cc:769] Sending connection context
0809 12:16:58.411684 (+ 219us) client_negotiation.cc:240] Negotiation successful
0809 12:16:58.412037 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":273,"threads_started":1}
I20260809 12:16:58.414775 26710 heartbeater.cc:347] Connected to a master server at 127.19.183.254:40023
I20260809 12:16:58.415035 26710 heartbeater.cc:464] Registering TS with master...
I20260809 12:16:58.415624 26710 heartbeater.cc:511] Master 127.19.183.254:40023 requested a full tablet report, sending...
I20260809 12:16:58.416978 26016 ts_manager.cc:194] Registered new tserver with Master: f34c1cc441784a619507b050435e07e7 (127.19.183.195:40153)
I20260809 12:16:58.417841 26016 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:53347
I20260809 12:16:58.426203 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:16:58.442955 26712 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:58.430574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56382 (local address 127.19.183.254:40023)
0809 12:16:58.430755 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:58.430759 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:58.431395 (+ 636us) server_negotiation.cc:407] Connection header received
0809 12:16:58.431604 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:58.431609 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:58.431664 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:58.431759 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:58.433151 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.433880 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.435154 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.435556 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.435694 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:58.437828 (+ 2134us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:58.437833 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:58.437837 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:58.437864 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:58.440461 (+ 2597us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:58.441082 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:58.441087 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:58.441092 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:58.441205 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:58.441558 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:58.441563 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:58.441566 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:58.441860 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:58.442107 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:58.442407 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 12:16:58.442640 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 12:16:58.446633 26016 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56382:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:16:58.453150 26016 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:16:58.477375 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:58.466618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54386 (local address 127.19.183.194:44863)
0809 12:16:58.467286 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:58.467293 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:58.467914 (+ 621us) server_negotiation.cc:407] Connection header received
0809 12:16:58.468283 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:58.468293 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:58.468521 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:58.468735 (+ 214us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:58.469802 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.471838 (+ 2036us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.474810 (+ 2972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.476087 (+ 1277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.476251 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:58.476440 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:58.476888 (+ 448us) server_negotiation.cc:299] Negotiation successful
0809 12:16:58.477047 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":508,"thread_start_us":386,"threads_started":1}
I20260809 12:16:58.477522 26727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:58.465633 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:44863 (local address 127.0.0.1:54386)
0809 12:16:58.467686 (+ 2053us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:58.467704 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:58.468027 (+ 323us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:58.469052 (+ 1025us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:58.469062 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:58.469111 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:58.469555 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:58.469590 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.472013 (+ 2423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.472022 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:58.474639 (+ 2617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:58.474655 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.476319 (+ 1664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.476328 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:58.476572 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:58.476670 (+ 98us) client_negotiation.cc:769] Sending connection context
0809 12:16:58.476818 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 12:16:58.477062 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1901,"mutex_wait_us":46,"thread_start_us":2064,"threads_started":1}
I20260809 12:16:58.480172 26730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:58.467922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:40735 (local address 127.0.0.1:41670)
0809 12:16:58.468634 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:58.468656 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:58.468856 (+ 200us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:58.473111 (+ 4255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:58.473116 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:58.473143 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:58.473672 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:58.473685 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.475455 (+ 1770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.475462 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:58.477434 (+ 1972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:58.477450 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.479387 (+ 1937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.479394 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:58.479560 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:58.479609 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 12:16:58.479721 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 12:16:58.479840 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"thread_start_us":2472,"threads_started":1}
I20260809 12:16:58.480260 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:58.470198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41670 (local address 127.19.183.193:40735)
0809 12:16:58.472588 (+ 2390us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:58.472597 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:58.472623 (+ 26us) server_negotiation.cc:407] Connection header received
0809 12:16:58.472728 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:58.472737 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:58.472913 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:58.473121 (+ 208us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:58.473877 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.475321 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.477652 (+ 2331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.479100 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.479363 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:58.479520 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:58.479783 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 12:16:58.479948 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2171,"thread_start_us":1986,"threads_started":1}
I20260809 12:16:58.481053 26726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:58.465487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:40153 (local address 127.0.0.1:42532)
0809 12:16:58.466194 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:58.466246 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:58.466413 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:58.473421 (+ 7008us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:58.473428 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:58.473459 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:58.474072 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:58.474086 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.476463 (+ 2377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.476468 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:58.478385 (+ 1917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:58.478420 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.480561 (+ 2141us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.480566 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:58.480674 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:58.480693 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 12:16:58.480750 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 12:16:58.480823 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":33,"thread_start_us":336,"threads_started":1}
I20260809 12:16:58.481729 26728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:58.465997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42532 (local address 127.19.183.195:40153)
0809 12:16:58.472867 (+ 6870us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:58.472876 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:58.472908 (+ 32us) server_negotiation.cc:407] Connection header received
0809 12:16:58.473003 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:58.473011 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:58.473231 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:58.473443 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:58.474341 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.476278 (+ 1937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.478592 (+ 2314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.480061 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.480265 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:58.481350 (+ 1085us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:58.481409 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 12:16:58.481503 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":6699,"thread_start_us":6542,"threads_started":1}
I20260809 12:16:58.482075 26395 tablet_service.cc:1467] Processing CreateTablet for tablet 57723f1f78454382bcda78df48f13f0c (DEFAULT_TABLE table=test-table [id=b9d751327406436cb65d7badbe64c07e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:58.483964 26395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57723f1f78454382bcda78df48f13f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:58.484086 26191 tablet_service.cc:1467] Processing CreateTablet for tablet 57723f1f78454382bcda78df48f13f0c (DEFAULT_TABLE table=test-table [id=b9d751327406436cb65d7badbe64c07e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:58.485051 26594 tablet_service.cc:1467] Processing CreateTablet for tablet 57723f1f78454382bcda78df48f13f0c (DEFAULT_TABLE table=test-table [id=b9d751327406436cb65d7badbe64c07e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:16:58.485502 26191 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57723f1f78454382bcda78df48f13f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:58.486093 26594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 57723f1f78454382bcda78df48f13f0c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:58.498018 26732 tablet_bootstrap.cc:492] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7: Bootstrap starting.
I20260809 12:16:58.500840 26734 tablet_bootstrap.cc:492] T 57723f1f78454382bcda78df48f13f0c P b83194fddfa346099ceeff7e81b16266: Bootstrap starting.
I20260809 12:16:58.502144 26732 tablet_bootstrap.cc:654] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:58.502620 26733 tablet_bootstrap.cc:492] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374: Bootstrap starting.
I20260809 12:16:58.503597 26732 log.cc:824] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7: Log is configured to *not* fsync() on all Append() calls
I20260809 12:16:58.504866 26734 tablet_bootstrap.cc:654] T 57723f1f78454382bcda78df48f13f0c P b83194fddfa346099ceeff7e81b16266: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:58.506110 26733 tablet_bootstrap.cc:654] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:58.506326 26734 log.cc:824] T 57723f1f78454382bcda78df48f13f0c P b83194fddfa346099ceeff7e81b16266: Log is configured to *not* fsync() on all Append() calls
I20260809 12:16:58.507331 26732 tablet_bootstrap.cc:492] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7: No bootstrap required, opened a new log
I20260809 12:16:58.507362 26733 log.cc:824] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374: Log is configured to *not* fsync() on all Append() calls
I20260809 12:16:58.507606 26732 ts_tablet_manager.cc:1397] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260809 12:16:58.509513 26734 tablet_bootstrap.cc:492] T 57723f1f78454382bcda78df48f13f0c P b83194fddfa346099ceeff7e81b16266: No bootstrap required, opened a new log
I20260809 12:16:58.509843 26734 ts_tablet_manager.cc:1397] T 57723f1f78454382bcda78df48f13f0c P b83194fddfa346099ceeff7e81b16266: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.008s
I20260809 12:16:58.510335 26733 tablet_bootstrap.cc:492] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374: No bootstrap required, opened a new log
I20260809 12:16:58.510579 26733 ts_tablet_manager.cc:1397] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 12:16:58.518321 26732 raft_consensus.cc:348] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f34c1cc441784a619507b050435e07e7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 40153 } } peers { permanent_uuid: "ed350d3b6a66454f9fb325d13b31d374" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 44863 } } peers { permanent_uuid: "b83194fddfa346099ceeff7e81b16266" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40735 } }
I20260809 12:16:58.518921 26732 raft_consensus.cc:374] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:58.519075 26732 raft_consensus.cc:729] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f34c1cc441784a619507b050435e07e7, State: Initialized, Role: FOLLOWER
I20260809 12:16:58.519972 26732 consensus_queue.cc:260] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f34c1cc441784a619507b050435e07e7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 40153 } } peers { permanent_uuid: "ed350d3b6a66454f9fb325d13b31d374" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 44863 } } peers { permanent_uuid: "b83194fddfa346099ceeff7e81b16266" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40735 } }
I20260809 12:16:58.520479 26734 raft_consensus.cc:348] T 57723f1f78454382bcda78df48f13f0c P b83194fddfa346099ceeff7e81b16266 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f34c1cc441784a619507b050435e07e7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 40153 } } peers { permanent_uuid: "ed350d3b6a66454f9fb325d13b31d374" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 44863 } } peers { permanent_uuid: "b83194fddfa346099ceeff7e81b16266" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40735 } }
I20260809 12:16:58.521088 26734 raft_consensus.cc:374] T 57723f1f78454382bcda78df48f13f0c P b83194fddfa346099ceeff7e81b16266 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:58.521270 26734 raft_consensus.cc:729] T 57723f1f78454382bcda78df48f13f0c P b83194fddfa346099ceeff7e81b16266 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b83194fddfa346099ceeff7e81b16266, State: Initialized, Role: FOLLOWER
I20260809 12:16:58.522044 26734 consensus_queue.cc:260] T 57723f1f78454382bcda78df48f13f0c P b83194fddfa346099ceeff7e81b16266 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f34c1cc441784a619507b050435e07e7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 40153 } } peers { permanent_uuid: "ed350d3b6a66454f9fb325d13b31d374" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 44863 } } peers { permanent_uuid: "b83194fddfa346099ceeff7e81b16266" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40735 } }
I20260809 12:16:58.524473 26733 raft_consensus.cc:348] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f34c1cc441784a619507b050435e07e7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 40153 } } peers { permanent_uuid: "ed350d3b6a66454f9fb325d13b31d374" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 44863 } } peers { permanent_uuid: "b83194fddfa346099ceeff7e81b16266" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40735 } }
I20260809 12:16:58.525002 26733 raft_consensus.cc:374] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:58.525125 26733 raft_consensus.cc:729] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed350d3b6a66454f9fb325d13b31d374, State: Initialized, Role: FOLLOWER
I20260809 12:16:58.525853 26733 consensus_queue.cc:260] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f34c1cc441784a619507b050435e07e7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 40153 } } peers { permanent_uuid: "ed350d3b6a66454f9fb325d13b31d374" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 44863 } } peers { permanent_uuid: "b83194fddfa346099ceeff7e81b16266" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40735 } }
I20260809 12:16:58.529011 26495 heartbeater.cc:503] Master 127.19.183.254:40023 was elected leader, sending a full tablet report...
I20260809 12:16:58.529505 26733 ts_tablet_manager.cc:1428] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374: Time spent starting tablet: real 0.019s user 0.016s sys 0.000s
I20260809 12:16:58.531348 26741 raft_consensus.cc:482] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:16:58.531430 26710 heartbeater.cc:503] Master 127.19.183.254:40023 was elected leader, sending a full tablet report...
I20260809 12:16:58.531628 26741 raft_consensus.cc:504] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f34c1cc441784a619507b050435e07e7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 40153 } } peers { permanent_uuid: "ed350d3b6a66454f9fb325d13b31d374" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 44863 } } peers { permanent_uuid: "b83194fddfa346099ceeff7e81b16266" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40735 } }
I20260809 12:16:58.533617 26741 leader_election.cc:288] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed350d3b6a66454f9fb325d13b31d374 (127.19.183.194:44863), b83194fddfa346099ceeff7e81b16266 (127.19.183.193:40735)
I20260809 12:16:58.537989 26732 ts_tablet_manager.cc:1428] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7: Time spent starting tablet: real 0.030s user 0.009s sys 0.009s
I20260809 12:16:58.540251 26734 ts_tablet_manager.cc:1428] T 57723f1f78454382bcda78df48f13f0c P b83194fddfa346099ceeff7e81b16266: Time spent starting tablet: real 0.030s user 0.013s sys 0.002s
I20260809 12:16:58.544798 26744 raft_consensus.cc:482] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:16:58.545135 26744 raft_consensus.cc:504] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f34c1cc441784a619507b050435e07e7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 40153 } } peers { permanent_uuid: "ed350d3b6a66454f9fb325d13b31d374" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 44863 } } peers { permanent_uuid: "b83194fddfa346099ceeff7e81b16266" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40735 } }
I20260809 12:16:58.547696 26713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:58.534919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:44863 (local address 127.19.183.195:44583)
0809 12:16:58.535253 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:58.535278 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:58.535408 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:58.536323 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:58.536332 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:58.536376 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:58.536618 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:58.536634 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.538086 (+ 1452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.538092 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:58.542344 (+ 4252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:58.542365 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.546656 (+ 4291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.546665 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:58.546961 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:58.547027 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 12:16:58.547140 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 12:16:58.547329 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137}
I20260809 12:16:58.548785 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:58.535534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:44583 (local address 127.19.183.194:44863)
0809 12:16:58.535780 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:58.535789 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:58.535813 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:16:58.535892 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:58.535901 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:58.535992 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:58.536103 (+ 111us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:58.536804 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.537900 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.542554 (+ 4654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.544329 (+ 1775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.544516 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:58.544633 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:58.548206 (+ 3573us) server_negotiation.cc:299] Negotiation successful
0809 12:16:58.548405 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64,"spinlock_wait_cycles":5376}
I20260809 12:16:58.550148 26419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57723f1f78454382bcda78df48f13f0c" candidate_uuid: "f34c1cc441784a619507b050435e07e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed350d3b6a66454f9fb325d13b31d374" is_pre_election: true
I20260809 12:16:58.550742 26419 raft_consensus.cc:2454] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f34c1cc441784a619507b050435e07e7 in term 0.
I20260809 12:16:58.551651 26522 leader_election.cc:302] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed350d3b6a66454f9fb325d13b31d374, f34c1cc441784a619507b050435e07e7; no voters:
I20260809 12:16:58.552178 26739 raft_consensus.cc:2781] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:16:58.552374 26739 raft_consensus.cc:482] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:16:58.552569 26739 raft_consensus.cc:3037] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:58.557085 26739 raft_consensus.cc:504] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f34c1cc441784a619507b050435e07e7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 40153 } } peers { permanent_uuid: "ed350d3b6a66454f9fb325d13b31d374" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 44863 } } peers { permanent_uuid: "b83194fddfa346099ceeff7e81b16266" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40735 } }
I20260809 12:16:58.558580 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:58.539152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:46761 (local address 127.19.183.193:40735)
0809 12:16:58.539350 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:58.539356 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:58.549536 (+ 10180us) server_negotiation.cc:407] Connection header received
0809 12:16:58.549859 (+ 323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:58.549868 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:58.549963 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:58.550059 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:58.550831 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.552187 (+ 1356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.555301 (+ 3114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.556940 (+ 1639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.557116 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:58.557218 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:58.558046 (+ 828us) server_negotiation.cc:299] Negotiation successful
0809 12:16:58.558229 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 12:16:58.558440 26419 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57723f1f78454382bcda78df48f13f0c" candidate_uuid: "f34c1cc441784a619507b050435e07e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed350d3b6a66454f9fb325d13b31d374"
I20260809 12:16:58.558787 26419 raft_consensus.cc:3037] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:58.559805 26713 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:58.539332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:40735 (local address 127.19.183.195:46761)
0809 12:16:58.549425 (+ 10093us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:58.549449 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:58.549660 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:58.550319 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:58.550326 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:58.550359 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:58.550618 (+ 259us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:58.550631 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.552408 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.552416 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:58.554989 (+ 2573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:58.555057 (+ 68us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.557442 (+ 2385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.557451 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:58.557800 (+ 349us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:58.557849 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 12:16:58.559183 (+ 1334us) client_negotiation.cc:240] Negotiation successful
0809 12:16:58.559385 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9960,"threads_started":1}
I20260809 12:16:58.561426 26217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57723f1f78454382bcda78df48f13f0c" candidate_uuid: "f34c1cc441784a619507b050435e07e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b83194fddfa346099ceeff7e81b16266" is_pre_election: true
I20260809 12:16:58.561952 26217 raft_consensus.cc:2454] T 57723f1f78454382bcda78df48f13f0c P b83194fddfa346099ceeff7e81b16266 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f34c1cc441784a619507b050435e07e7 in term 0.
I20260809 12:16:58.563027 26739 leader_election.cc:288] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7 [CANDIDATE]: Term 1 election: Requested vote from peers ed350d3b6a66454f9fb325d13b31d374 (127.19.183.194:44863), b83194fddfa346099ceeff7e81b16266 (127.19.183.193:40735)
I20260809 12:16:58.563387 26744 leader_election.cc:288] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f34c1cc441784a619507b050435e07e7 (127.19.183.195:40153), b83194fddfa346099ceeff7e81b16266 (127.19.183.193:40735)
I20260809 12:16:58.564504 26217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57723f1f78454382bcda78df48f13f0c" candidate_uuid: "f34c1cc441784a619507b050435e07e7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b83194fddfa346099ceeff7e81b16266"
I20260809 12:16:58.564694 26217 raft_consensus.cc:3037] T 57723f1f78454382bcda78df48f13f0c P b83194fddfa346099ceeff7e81b16266 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:58.566224 26419 raft_consensus.cc:2454] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f34c1cc441784a619507b050435e07e7 in term 1.
I20260809 12:16:58.566664 26217 raft_consensus.cc:2454] T 57723f1f78454382bcda78df48f13f0c P b83194fddfa346099ceeff7e81b16266 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f34c1cc441784a619507b050435e07e7 in term 1.
I20260809 12:16:58.567243 26527 leader_election.cc:302] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b83194fddfa346099ceeff7e81b16266, f34c1cc441784a619507b050435e07e7; no voters:
I20260809 12:16:58.567574 26739 raft_consensus.cc:2781] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:16:58.567739 26739 raft_consensus.cc:686] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7 [term 1 LEADER]: Becoming Leader. State: Replica: f34c1cc441784a619507b050435e07e7, State: Running, Role: LEADER
I20260809 12:16:58.568820 26739 consensus_queue.cc:237] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f34c1cc441784a619507b050435e07e7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 40153 } } peers { permanent_uuid: "ed350d3b6a66454f9fb325d13b31d374" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 44863 } } peers { permanent_uuid: "b83194fddfa346099ceeff7e81b16266" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40735 } }
I20260809 12:16:58.575148 26728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:58.547863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:44791 (local address 127.19.183.195:40153)
0809 12:16:58.548045 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:58.548053 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:58.564234 (+ 16181us) server_negotiation.cc:407] Connection header received
0809 12:16:58.566995 (+ 2761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:58.567000 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:58.567085 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:58.567175 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:58.568049 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.569099 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.571674 (+ 2575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.573431 (+ 1757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.573616 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:58.573718 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:58.574565 (+ 847us) server_negotiation.cc:299] Negotiation successful
0809 12:16:58.574782 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 12:16:58.576284 26746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:58.547417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:40153 (local address 127.19.183.194:44791)
0809 12:16:58.564138 (+ 16721us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:58.564157 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:58.564334 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:58.567383 (+ 3049us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:58.567387 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:58.567416 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:58.567867 (+ 451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:58.567880 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.569277 (+ 1397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.569282 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:58.571478 (+ 2196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:58.571500 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.574050 (+ 2550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.574060 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:58.574317 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:58.574369 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 12:16:58.575567 (+ 1198us) client_negotiation.cc:240] Negotiation successful
0809 12:16:58.575761 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16595,"thread_start_us":17008,"threads_started":1}
I20260809 12:16:58.577458 26630 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57723f1f78454382bcda78df48f13f0c" candidate_uuid: "ed350d3b6a66454f9fb325d13b31d374" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f34c1cc441784a619507b050435e07e7" is_pre_election: true
I20260809 12:16:58.579223 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:58.564224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:37047 (local address 127.19.183.193:40735)
0809 12:16:58.564410 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:58.564417 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:58.569069 (+ 4652us) server_negotiation.cc:407] Connection header received
0809 12:16:58.569297 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:58.569304 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:58.569389 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:58.569475 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:16:58.570487 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.571838 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.574928 (+ 3090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.576456 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.576616 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:58.576706 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:58.578681 (+ 1975us) server_negotiation.cc:299] Negotiation successful
0809 12:16:58.578887 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
W20260809 12:16:58.579937 26293 tablet.cc:2369] T 57723f1f78454382bcda78df48f13f0c P b83194fddfa346099ceeff7e81b16266: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:16:58.580119 26748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:58.563687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:40735 (local address 127.19.183.194:37047)
0809 12:16:58.568964 (+ 5277us) negotiation.cc:107] Waiting for socket to connect
0809 12:16:58.568984 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:16:58.569162 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:16:58.569774 (+ 612us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:16:58.569778 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:16:58.569807 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:16:58.570299 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:58.570324 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.572078 (+ 1754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.572086 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:58.574608 (+ 2522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:16:58.574627 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.576898 (+ 2271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.576906 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:16:58.578429 (+ 1523us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:58.578485 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 12:16:58.579486 (+ 1001us) client_negotiation.cc:240] Negotiation successful
0809 12:16:58.579680 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5121,"thread_start_us":4936,"threads_started":1}
I20260809 12:16:58.580842 26217 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "57723f1f78454382bcda78df48f13f0c" candidate_uuid: "ed350d3b6a66454f9fb325d13b31d374" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b83194fddfa346099ceeff7e81b16266" is_pre_election: true
I20260809 12:16:58.581370 26217 raft_consensus.cc:2379] T 57723f1f78454382bcda78df48f13f0c P b83194fddfa346099ceeff7e81b16266 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate ed350d3b6a66454f9fb325d13b31d374 in current term 1: Already voted for candidate f34c1cc441784a619507b050435e07e7 in this term.
I20260809 12:16:58.582218 26329 leader_election.cc:302] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: ed350d3b6a66454f9fb325d13b31d374; no voters: b83194fddfa346099ceeff7e81b16266, f34c1cc441784a619507b050435e07e7
I20260809 12:16:58.582897 26744 raft_consensus.cc:2726] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 12:16:58.586874 26016 catalog_manager.cc:5495] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7 reported cstate change: term changed from 0 to 1, leader changed from <none> to f34c1cc441784a619507b050435e07e7 (127.19.183.195). New cstate: current_term: 1 leader_uuid: "f34c1cc441784a619507b050435e07e7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f34c1cc441784a619507b050435e07e7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 40153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ed350d3b6a66454f9fb325d13b31d374" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 44863 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b83194fddfa346099ceeff7e81b16266" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40735 } health_report { overall_health: UNKNOWN } } }
I20260809 12:16:58.609606 26728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:58.602744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42546 (local address 127.19.183.195:40153)
0809 12:16:58.602955 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:58.602964 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:58.603011 (+ 47us) server_negotiation.cc:407] Connection header received
0809 12:16:58.603233 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:58.603241 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:58.603379 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:58.603573 (+ 194us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:16:58.603884 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.605231 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.606534 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.607353 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.607524 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:58.607938 (+ 414us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:16:58.608798 (+ 860us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:16:58.608967 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:58.609024 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 12:16:58.609134 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
W20260809 12:16:58.611743 26496 tablet.cc:2369] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:16:58.617607 26419 raft_consensus.cc:1261] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374 [term 1 FOLLOWER]: Refusing update from remote peer f34c1cc441784a619507b050435e07e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:16:58.617848 26217 raft_consensus.cc:1261] T 57723f1f78454382bcda78df48f13f0c P b83194fddfa346099ceeff7e81b16266 [term 1 FOLLOWER]: Refusing update from remote peer f34c1cc441784a619507b050435e07e7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:16:58.618552 26741 consensus_queue.cc:1035] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed350d3b6a66454f9fb325d13b31d374" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 44863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:16:58.618973 26739 consensus_queue.cc:1035] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b83194fddfa346099ceeff7e81b16266" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40735 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:16:58.637661 26757 mvcc.cc:205] Tried to move back new op lower bound from 7316593945048231936 to 7316593944862543872. Current Snapshot: MvccSnapshot[applied={T|T < 7316593945048231936}]
I20260809 12:16:58.639276 26755 mvcc.cc:205] Tried to move back new op lower bound from 7316593945048231936 to 7316593944862543872. Current Snapshot: MvccSnapshot[applied={T|T < 7316593945048231936}]
I20260809 12:16:58.655761 26016 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:16:58.656050 26016 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56382:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:16:58.660418 26016 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 81aa21b2439244979b942f89e48ff349: Sending DeleteTablet for 3 replicas of tablet 57723f1f78454382bcda78df48f13f0c
I20260809 12:16:58.661571 26594 tablet_service.cc:1514] Processing DeleteTablet for tablet 57723f1f78454382bcda78df48f13f0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:16:58 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:42532
I20260809 12:16:58.661589 26395 tablet_service.cc:1514] Processing DeleteTablet for tablet 57723f1f78454382bcda78df48f13f0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:16:58 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:54386
I20260809 12:16:58.666877 26191 tablet_service.cc:1514] Processing DeleteTablet for tablet 57723f1f78454382bcda78df48f13f0c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:16:58 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:41670
I20260809 12:16:58.668300 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 26090
I20260809 12:16:58.681965 26768 tablet_replica.cc:330] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7: stopping tablet replica
I20260809 12:16:58.682497 26768 raft_consensus.cc:2229] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:16:58.683257 26768 raft_consensus.cc:2258] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:16:58.691341 26771 tablet_replica.cc:330] T 57723f1f78454382bcda78df48f13f0c P b83194fddfa346099ceeff7e81b16266: stopping tablet replica
I20260809 12:16:58.692159 26771 raft_consensus.cc:2229] T 57723f1f78454382bcda78df48f13f0c P b83194fddfa346099ceeff7e81b16266 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:16:58.692303 26767 tablet_replica.cc:330] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374: stopping tablet replica
I20260809 12:16:58.692711 26767 raft_consensus.cc:2229] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:16:58.692904 26771 raft_consensus.cc:2258] T 57723f1f78454382bcda78df48f13f0c P b83194fddfa346099ceeff7e81b16266 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:16:58.693229 26767 raft_consensus.cc:2258] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:16:58.696100 26771 ts_tablet_manager.cc:1905] T 57723f1f78454382bcda78df48f13f0c P b83194fddfa346099ceeff7e81b16266: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:16:58.699792 26768 ts_tablet_manager.cc:1905] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 12:16:58 dist-test-slave-39l8 krb5kdc[25950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.193@KRBTEST.COM
I20260809 12:16:58.702020 26767 ts_tablet_manager.cc:1905] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:16:58.709177 26767 ts_tablet_manager.cc:1918] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:16:58.709371 26767 log.cc:1196] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/57723f1f78454382bcda78df48f13f0c
I20260809 12:16:58.709945 26767 ts_tablet_manager.cc:1939] T 57723f1f78454382bcda78df48f13f0c P ed350d3b6a66454f9fb325d13b31d374: Deleting consensus metadata
I20260809 12:16:58.711066 26731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:58.670123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41676 (local address 127.19.183.193:40735)
0809 12:16:58.670348 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:58.670357 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:58.687858 (+ 17501us) server_negotiation.cc:407] Connection header received
0809 12:16:58.688201 (+ 343us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:58.688210 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:58.688345 (+ 135us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:58.688449 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:58.693423 (+ 4974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.694617 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.696506 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.697100 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.697195 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:58.702998 (+ 5803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:58.703012 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:58.703030 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:58.703073 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:58.707166 (+ 4093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:58.708058 (+ 892us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:58.708068 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:58.708072 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:58.708234 (+ 162us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:58.708719 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:58.708723 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:58.708726 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:58.709198 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:58.709723 (+ 525us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:58.710421 (+ 698us) server_negotiation.cc:299] Negotiation successful
0809 12:16:58.710684 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:16:58.712978 26281 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:16:58.716259 25996 catalog_manager.cc:4841] TS ed350d3b6a66454f9fb325d13b31d374 (127.19.183.194:44863): tablet 57723f1f78454382bcda78df48f13f0c (table test-table [id=b9d751327406436cb65d7badbe64c07e]) successfully deleted
I20260809 12:16:58.719741 26768 ts_tablet_manager.cc:1918] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:16:58.720099 26768 log.cc:1196] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/57723f1f78454382bcda78df48f13f0c
I20260809 12:16:58.721400 26768 ts_tablet_manager.cc:1939] T 57723f1f78454382bcda78df48f13f0c P f34c1cc441784a619507b050435e07e7: Deleting consensus metadata
I20260809 12:16:58.723599 25995 catalog_manager.cc:4841] TS f34c1cc441784a619507b050435e07e7 (127.19.183.195:40153): tablet 57723f1f78454382bcda78df48f13f0c (table test-table [id=b9d751327406436cb65d7badbe64c07e]) successfully deleted
I20260809 12:16:58.878021 26771 ts_tablet_manager.cc:1918] T 57723f1f78454382bcda78df48f13f0c P b83194fddfa346099ceeff7e81b16266: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:16:58.878430 26771 log.cc:1196] T 57723f1f78454382bcda78df48f13f0c P b83194fddfa346099ceeff7e81b16266: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/57723f1f78454382bcda78df48f13f0c
I20260809 12:16:58.879096 26771 ts_tablet_manager.cc:1939] T 57723f1f78454382bcda78df48f13f0c P b83194fddfa346099ceeff7e81b16266: Deleting consensus metadata
I20260809 12:16:58.881266 25998 catalog_manager.cc:4841] TS b83194fddfa346099ceeff7e81b16266 (127.19.183.193:40735): tablet 57723f1f78454382bcda78df48f13f0c (table test-table [id=b9d751327406436cb65d7badbe64c07e]) successfully deleted
I20260809 12:16:58.882074 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 26090
I20260809 12:16:58.919261 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 26300
Aug 09 12:16:58 dist-test-slave-39l8 krb5kdc[25950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.194@KRBTEST.COM
I20260809 12:16:58.935364 26729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:58.920054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54402 (local address 127.19.183.194:44863)
0809 12:16:58.920303 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:58.920309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:58.920333 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:16:58.920434 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:58.920440 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:58.920536 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:58.920647 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:58.922060 (+ 1413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.923173 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.924400 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:58.925025 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:58.925171 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:58.929677 (+ 4506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:58.929690 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:58.929707 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:58.929747 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:58.932616 (+ 2869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:58.933356 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:58.933363 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:58.933365 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:58.933481 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:58.933892 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:58.933895 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:58.933897 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:58.934239 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:58.934468 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:58.934857 (+ 389us) server_negotiation.cc:299] Negotiation successful
0809 12:16:58.935026 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260809 12:16:58.936017 26483 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:16:59.028483 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 26300
I20260809 12:16:59.056207 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 26501
Aug 09 12:16:59 dist-test-slave-39l8 krb5kdc[25950](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277816, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.195@KRBTEST.COM
I20260809 12:16:59.074182 26728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:59.057118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42556 (local address 127.19.183.195:40153)
0809 12:16:59.057291 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:59.057295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:59.057462 (+ 167us) server_negotiation.cc:407] Connection header received
0809 12:16:59.057698 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:59.057704 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:59.057774 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:59.057884 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:59.059275 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:59.060025 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:59.061522 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:16:59.062013 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:16:59.062171 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:16:59.066696 (+ 4525us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:59.066733 (+ 37us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:59.066755 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:16:59.066807 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:59.071153 (+ 4346us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:59.071785 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:59.071797 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:59.071800 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:59.071969 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:16:59.072368 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:16:59.072373 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:16:59.072377 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:16:59.072876 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:59.073220 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:59.073540 (+ 320us) server_negotiation.cc:299] Negotiation successful
0809 12:16:59.073795 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 12:16:59.074977 26703 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:16:59.195367 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 26501
I20260809 12:16:59.223551 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 25975
I20260809 12:16:59.224718 26054 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:16:59.333503 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 25975
2026-08-09T12:16:59Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4520 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T12:16:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:16:59Z Disabled control of system clock
I20260809 12:16:59.417441 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:41093
--webserver_interface=127.19.183.254
--webserver_port=0
--builtin_ntp_servers=127.19.183.212:42507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:41093
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 12:16:59.740793 26786 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:16:59.741110 26786 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:16:59.741248 26786 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:16:59.748555 26786 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:16:59.748656 26786 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:16:59.748693 26786 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:16:59.748750 26786 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:16:59.748793 26786 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:16:59.757230 26786 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:42507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:41093
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:41093
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:16:59.758399 26786 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:16:59.760229 26786 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:16:59.772032 26796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:59.772079 26795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:16:59.771977 26798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:16:59.773098 26786 server_base.cc:1034] running on GCE node
I20260809 12:16:59.773850 26786 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:16:59.775265 26786 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:16:59.776465 26786 hybrid_clock.cc:648] HybridClock initialized: now 1786277819776424 us; error 43 us; skew 500 ppm
I20260809 12:16:59.776750 26786 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:16:59.778918 26786 webserver.cc:460] Webserver started at http://127.19.183.254:43243/ using document root <none> and password file <none>
I20260809 12:16:59.779559 26786 fs_manager.cc:359] Metadata directory not provided
I20260809 12:16:59.779656 26786 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:16:59.779904 26786 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:16:59.782548 26786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "707e5cddf8c74405b6fe27760e6ddc6b"
format_stamp: "Formatted at 2026-08-09 12:16:59 on dist-test-slave-39l8"
server_key: "a805892d312ca7ed7b0e09872656f0f5"
server_key_iv: "c9b99025a79781412db360299749995d"
server_key_version: "encryptionkey@0"
I20260809 12:16:59.783370 26786 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "707e5cddf8c74405b6fe27760e6ddc6b"
format_stamp: "Formatted at 2026-08-09 12:16:59 on dist-test-slave-39l8"
server_key: "a805892d312ca7ed7b0e09872656f0f5"
server_key_iv: "c9b99025a79781412db360299749995d"
server_key_version: "encryptionkey@0"
I20260809 12:16:59.788185 26786 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 12:16:59.791486 26808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:16:59.792781 26786 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:16:59.792920 26786 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "707e5cddf8c74405b6fe27760e6ddc6b"
format_stamp: "Formatted at 2026-08-09 12:16:59 on dist-test-slave-39l8"
server_key: "a805892d312ca7ed7b0e09872656f0f5"
server_key_iv: "c9b99025a79781412db360299749995d"
server_key_version: "encryptionkey@0"
I20260809 12:16:59.793089 26786 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:16:59.798563 26786 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:16:59.799531 26786 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:16:59.799845 26786 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:16:59.818221 26786 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:41093
I20260809 12:16:59.818266 26880 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:41093 every 8 connection(s)
I20260809 12:16:59.819633 26786 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:16:59.823374 26881 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:16:59.826215 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 26786
I20260809 12:16:59.826439 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:16:59.827322 20191 external_mini_cluster.cc:1428] Setting key 822fa3071b068dc7512423ad0c7cdadf
I20260809 12:16:59.835661 26885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:16:59.829438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59288 (local address 127.19.183.254:41093)
0809 12:16:59.830469 (+ 1031us) server_negotiation.cc:206] Beginning negotiation
0809 12:16:59.830480 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:16:59.830522 (+ 42us) server_negotiation.cc:407] Connection header received
0809 12:16:59.832232 (+ 1710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:16:59.832254 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:16:59.832659 (+ 405us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:16:59.832969 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:16:59.834104 (+ 1135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:16:59.834110 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:16:59.834133 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:16:59.834147 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:16:59.834206 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:16:59.834222 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:16:59.834394 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 12:16:59.834538 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 12:16:59.834820 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":387,"threads_started":1}
I20260809 12:16:59.836254 26881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b: Bootstrap starting.
I20260809 12:16:59.839330 26881 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b: Neither blocks nor log segments found. Creating new log.
I20260809 12:16:59.840320 26881 log.cc:824] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b: Log is configured to *not* fsync() on all Append() calls
I20260809 12:16:59.843164 26881 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b: No bootstrap required, opened a new log
I20260809 12:16:59.850214 26881 raft_consensus.cc:348] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707e5cddf8c74405b6fe27760e6ddc6b" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41093 } }
I20260809 12:16:59.850551 26881 raft_consensus.cc:374] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:16:59.850662 26881 raft_consensus.cc:729] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 707e5cddf8c74405b6fe27760e6ddc6b, State: Initialized, Role: FOLLOWER
I20260809 12:16:59.851277 26881 consensus_queue.cc:260] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707e5cddf8c74405b6fe27760e6ddc6b" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41093 } }
I20260809 12:16:59.851507 26881 raft_consensus.cc:388] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:16:59.851660 26881 raft_consensus.cc:482] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:16:59.851836 26881 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:16:59.854184 26881 raft_consensus.cc:504] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707e5cddf8c74405b6fe27760e6ddc6b" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41093 } }
I20260809 12:16:59.854678 26881 leader_election.cc:302] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 707e5cddf8c74405b6fe27760e6ddc6b; no voters:
I20260809 12:16:59.855319 26881 leader_election.cc:288] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:16:59.855474 26887 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:16:59.856206 26887 raft_consensus.cc:686] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b [term 1 LEADER]: Becoming Leader. State: Replica: 707e5cddf8c74405b6fe27760e6ddc6b, State: Running, Role: LEADER
I20260809 12:16:59.856724 26887 consensus_queue.cc:237] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707e5cddf8c74405b6fe27760e6ddc6b" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41093 } }
I20260809 12:16:59.857353 26881 sys_catalog.cc:564] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:16:59.859516 26889 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 707e5cddf8c74405b6fe27760e6ddc6b. Latest consensus state: current_term: 1 leader_uuid: "707e5cddf8c74405b6fe27760e6ddc6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707e5cddf8c74405b6fe27760e6ddc6b" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41093 } } }
I20260809 12:16:59.859629 26888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "707e5cddf8c74405b6fe27760e6ddc6b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "707e5cddf8c74405b6fe27760e6ddc6b" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41093 } } }
I20260809 12:16:59.860023 26888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b [sys.catalog]: This master's current role is: LEADER
I20260809 12:16:59.860019 26889 sys_catalog.cc:458] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b [sys.catalog]: This master's current role is: LEADER
I20260809 12:16:59.863790 26899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:16:59.868768 26899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:16:59.877307 26899 catalog_manager.cc:1347] Generated new cluster ID: b1827ed0f1c442638c16c3a82dd5ddba
I20260809 12:16:59.877414 26899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:16:59.917447 26899 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:16:59.918011 26899 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:16:59.926491 26899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b: Generated new TSK 0
I20260809 12:16:59.927301 26899 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:16:59.937638 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:41093
--builtin_ntp_servers=127.19.183.212:42507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 12:17:00.294392 26915 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:00.294665 26915 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:00.294817 26915 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:00.301976 26915 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:00.302065 26915 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:00.302167 26915 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:17:00.311028 26915 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:42507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:41093
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:00.312235 26915 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:00.313895 26915 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:00.327375 26923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:00.327407 26920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:00.327438 26921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:00.328559 26915 server_base.cc:1034] running on GCE node
I20260809 12:17:00.329087 26915 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:00.329912 26915 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:00.331104 26915 hybrid_clock.cc:648] HybridClock initialized: now 1786277820331056 us; error 45 us; skew 500 ppm
I20260809 12:17:00.331370 26915 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:17:00.333540 26915 webserver.cc:460] Webserver started at http://127.19.183.193:33831/ using document root <none> and password file <none>
I20260809 12:17:00.334156 26915 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:00.334236 26915 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:00.334529 26915 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:00.337278 26915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "a320908b9b154baf9f3b0f9661265354"
format_stamp: "Formatted at 2026-08-09 12:17:00 on dist-test-slave-39l8"
server_key: "32fb9c0a6b8a1b3fedd1a3b9d838ac82"
server_key_iv: "d3fd2c609e4ec4006f7c022dab6e1244"
server_key_version: "encryptionkey@0"
I20260809 12:17:00.338107 26915 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "a320908b9b154baf9f3b0f9661265354"
format_stamp: "Formatted at 2026-08-09 12:17:00 on dist-test-slave-39l8"
server_key: "32fb9c0a6b8a1b3fedd1a3b9d838ac82"
server_key_iv: "d3fd2c609e4ec4006f7c022dab6e1244"
server_key_version: "encryptionkey@0"
I20260809 12:17:00.343218 26915 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 12:17:00.346417 26932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:00.347656 26915 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:17:00.347838 26915 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "a320908b9b154baf9f3b0f9661265354"
format_stamp: "Formatted at 2026-08-09 12:17:00 on dist-test-slave-39l8"
server_key: "32fb9c0a6b8a1b3fedd1a3b9d838ac82"
server_key_iv: "d3fd2c609e4ec4006f7c022dab6e1244"
server_key_version: "encryptionkey@0"
I20260809 12:17:00.348044 26915 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:00.353484 26915 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:00.354213 26915 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:00.354475 26915 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:00.355377 26915 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:00.356724 26915 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:00.356834 26915 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:00.356964 26915 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:00.357056 26915 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:00.390534 26915 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:36627
I20260809 12:17:00.390635 27081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:36627 every 8 connection(s)
I20260809 12:17:00.391880 26915 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:17:00.395776 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 26915
I20260809 12:17:00.395987 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:17:00.396723 20191 external_mini_cluster.cc:1428] Setting key 18d1b62041a03115c7fb8993f21286a8
I20260809 12:17:00.399634 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:41093
--builtin_ntp_servers=127.19.183.212:42507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 12:17:00.400377 27086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:00.394421 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:38171 (local address 127.19.183.254:41093)
0809 12:17:00.394906 (+ 485us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:00.394911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:00.396179 (+ 1268us) server_negotiation.cc:407] Connection header received
0809 12:17:00.398011 (+ 1832us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:00.398018 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:00.398089 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:00.398208 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:00.399591 (+ 1383us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:00.399599 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:00.399608 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:00.399623 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:00.399664 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:00.399675 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:00.399830 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:00.400082 (+ 252us) server_negotiation.cc:299] Negotiation successful
0809 12:17:00.400132 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":245,"threads_started":1}
I20260809 12:17:00.401693 27090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:00.395027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:41093 (local address 127.19.183.193:38171)
0809 12:17:00.395988 (+ 961us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:00.396024 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:00.397751 (+ 1727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:00.398449 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:00.398457 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:00.398545 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:00.399305 (+ 760us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:17:00.399307 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:00.399352 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:17:00.399432 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:00.399826 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:00.399842 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:00.399854 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 12:17:00.400144 (+ 290us) client_negotiation.cc:240] Negotiation successful
0809 12:17:00.400407 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"thread_start_us":303,"threads_started":1}
I20260809 12:17:00.403173 27083 heartbeater.cc:347] Connected to a master server at 127.19.183.254:41093
I20260809 12:17:00.403476 27083 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:00.404006 27083 heartbeater.cc:511] Master 127.19.183.254:41093 requested a full tablet report, sending...
I20260809 12:17:00.405946 26833 ts_manager.cc:194] Registered new tserver with Master: a320908b9b154baf9f3b0f9661265354 (127.19.183.193:36627)
W20260809 12:17:00.733992 27091 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:00.734289 27091 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:00.734436 27091 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:00.741219 27091 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:00.741349 27091 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:00.741511 27091 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:17:00.751286 27091 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:42507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:41093
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:00.752574 27091 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:00.754334 27091 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:00.768139 27098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:00.768229 27100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:00.768256 27103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:00.769120 27091 server_base.cc:1034] running on GCE node
I20260809 12:17:00.769744 27091 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:00.770617 27091 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:00.771843 27091 hybrid_clock.cc:648] HybridClock initialized: now 1786277820771789 us; error 47 us; skew 500 ppm
I20260809 12:17:00.772200 27091 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:17:00.774627 27091 webserver.cc:460] Webserver started at http://127.19.183.194:46327/ using document root <none> and password file <none>
I20260809 12:17:00.775345 27091 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:00.775475 27091 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:00.775822 27091 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:00.778482 27091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "259c08c43afc4a8583b72c96200d17d5"
format_stamp: "Formatted at 2026-08-09 12:17:00 on dist-test-slave-39l8"
server_key: "d4bc2ec96bfcdd7d86beb15824700aec"
server_key_iv: "42897555264b67c51cb429572bee5139"
server_key_version: "encryptionkey@0"
I20260809 12:17:00.779369 27091 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "259c08c43afc4a8583b72c96200d17d5"
format_stamp: "Formatted at 2026-08-09 12:17:00 on dist-test-slave-39l8"
server_key: "d4bc2ec96bfcdd7d86beb15824700aec"
server_key_iv: "42897555264b67c51cb429572bee5139"
server_key_version: "encryptionkey@0"
I20260809 12:17:00.784289 27091 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:17:00.787789 27111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:00.789006 27091 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:17:00.789193 27091 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "259c08c43afc4a8583b72c96200d17d5"
format_stamp: "Formatted at 2026-08-09 12:17:00 on dist-test-slave-39l8"
server_key: "d4bc2ec96bfcdd7d86beb15824700aec"
server_key_iv: "42897555264b67c51cb429572bee5139"
server_key_version: "encryptionkey@0"
I20260809 12:17:00.789420 27091 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:00.794814 27091 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:00.795576 27091 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:00.795852 27091 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:00.796865 27091 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:00.798334 27091 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:00.798466 27091 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:00.798611 27091 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:00.798703 27091 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:00.833448 27091 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:39847
I20260809 12:17:00.833500 27270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:39847 every 8 connection(s)
I20260809 12:17:00.834796 27091 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:17:00.835121 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 27091
I20260809 12:17:00.835289 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:17:00.836076 20191 external_mini_cluster.cc:1428] Setting key fe9604e341d6f757ac949b720e5a20c6
I20260809 12:17:00.839547 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:41093
--builtin_ntp_servers=127.19.183.212:42507
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 12:17:00.847997 27086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:00.837694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:45725 (local address 127.19.183.254:41093)
0809 12:17:00.837869 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:00.837875 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:00.843774 (+ 5899us) server_negotiation.cc:407] Connection header received
0809 12:17:00.845475 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:00.845483 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:00.845566 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:00.845688 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:00.847165 (+ 1477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:00.847174 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:00.847182 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:00.847205 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:00.847259 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:00.847278 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:00.847407 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:00.847588 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 12:17:00.847665 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 12:17:00.848769 27274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:00.838608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:41093 (local address 127.19.183.194:45725)
0809 12:17:00.843578 (+ 4970us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:00.843635 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:00.845211 (+ 1576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:00.845995 (+ 784us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:00.846006 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:00.846129 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:00.846860 (+ 731us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:17:00.846863 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:00.846901 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:17:00.846993 (+ 92us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:00.847394 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:00.847403 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:00.847409 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 12:17:00.847622 (+ 213us) client_negotiation.cc:240] Negotiation successful
0809 12:17:00.847810 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4427,"thread_start_us":4149,"threads_started":1}
I20260809 12:17:00.850142 27271 heartbeater.cc:347] Connected to a master server at 127.19.183.254:41093
I20260809 12:17:00.850459 27271 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:00.851197 27271 heartbeater.cc:511] Master 127.19.183.254:41093 requested a full tablet report, sending...
I20260809 12:17:00.852804 26833 ts_manager.cc:194] Registered new tserver with Master: 259c08c43afc4a8583b72c96200d17d5 (127.19.183.194:39847)
W20260809 12:17:01.177378 27276 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:01.177650 27276 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:01.177764 27276 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:01.184837 27276 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:01.184928 27276 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:01.185091 27276 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:17:01.194159 27276 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:42507
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:41093
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:01.195437 27276 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:01.197348 27276 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:01.210326 27287 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:01.210364 27284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:01.210884 27283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:01.211393 27276 server_base.cc:1034] running on GCE node
I20260809 12:17:01.211947 27276 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:01.212826 27276 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:01.214002 27276 hybrid_clock.cc:648] HybridClock initialized: now 1786277821213940 us; error 52 us; skew 500 ppm
I20260809 12:17:01.214313 27276 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:17:01.216509 27276 webserver.cc:460] Webserver started at http://127.19.183.195:35997/ using document root <none> and password file <none>
I20260809 12:17:01.217243 27276 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:01.217378 27276 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:01.217744 27276 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:01.220642 27276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "f83c4c167b984f42954217da3e34a6b7"
format_stamp: "Formatted at 2026-08-09 12:17:01 on dist-test-slave-39l8"
server_key: "9f8d579343fb656031209e7eef88ade2"
server_key_iv: "5f9f4060f6f3307eb9498f9153048a3c"
server_key_version: "encryptionkey@0"
I20260809 12:17:01.221505 27276 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "f83c4c167b984f42954217da3e34a6b7"
format_stamp: "Formatted at 2026-08-09 12:17:01 on dist-test-slave-39l8"
server_key: "9f8d579343fb656031209e7eef88ade2"
server_key_iv: "5f9f4060f6f3307eb9498f9153048a3c"
server_key_version: "encryptionkey@0"
I20260809 12:17:01.226562 27276 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 12:17:01.229997 27294 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:01.231372 27276 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:17:01.231549 27276 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "f83c4c167b984f42954217da3e34a6b7"
format_stamp: "Formatted at 2026-08-09 12:17:01 on dist-test-slave-39l8"
server_key: "9f8d579343fb656031209e7eef88ade2"
server_key_iv: "5f9f4060f6f3307eb9498f9153048a3c"
server_key_version: "encryptionkey@0"
I20260809 12:17:01.231776 27276 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:01.237491 27276 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:01.238296 27276 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:01.238564 27276 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:01.239492 27276 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:01.240875 27276 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:01.240989 27276 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:01.241115 27276 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:01.241207 27276 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:01.275380 27276 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:43159
I20260809 12:17:01.275476 27452 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:43159 every 8 connection(s)
I20260809 12:17:01.276757 27276 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:17:01.284473 27086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:01.279349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:41153 (local address 127.19.183.254:41093)
0809 12:17:01.279579 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:01.279586 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:01.280906 (+ 1320us) server_negotiation.cc:407] Connection header received
0809 12:17:01.282483 (+ 1577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:01.282487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:01.282549 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:01.282653 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:01.283885 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:01.283889 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:01.283892 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:01.283908 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:01.283938 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:01.283949 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:01.284043 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:01.284217 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 12:17:01.284266 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 12:17:01.285527 27456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:01.279903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:41093 (local address 127.19.183.195:41153)
0809 12:17:01.280747 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:01.280791 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:01.282252 (+ 1461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:01.282906 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:01.282913 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:01.282998 (+ 85us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:01.283610 (+ 612us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:17:01.283612 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:01.283647 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:17:01.283728 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:01.284034 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:01.284041 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:01.284046 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 12:17:01.284256 (+ 210us) client_negotiation.cc:240] Negotiation successful
0809 12:17:01.284438 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":249,"threads_started":1}
I20260809 12:17:01.285805 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 27276
I20260809 12:17:01.285971 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:17:01.286545 20191 external_mini_cluster.cc:1428] Setting key b5a77db969d14f4a1b0ab454c5a287c8
I20260809 12:17:01.286975 27453 heartbeater.cc:347] Connected to a master server at 127.19.183.254:41093
I20260809 12:17:01.287302 27453 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:01.287985 27453 heartbeater.cc:511] Master 127.19.183.254:41093 requested a full tablet report, sending...
I20260809 12:17:01.289574 26832 ts_manager.cc:194] Registered new tserver with Master: f83c4c167b984f42954217da3e34a6b7 (127.19.183.195:43159)
I20260809 12:17:01.301698 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:17:01.308991 27086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:01.306174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37596 (local address 127.19.183.254:41093)
0809 12:17:01.306336 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:01.306341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:01.306914 (+ 573us) server_negotiation.cc:407] Connection header received
0809 12:17:01.307133 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:01.307137 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:01.307190 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:01.307296 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:01.308306 (+ 1010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:01.308311 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:01.308316 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:01.308370 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:01.308401 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:01.308413 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:01.308543 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:01.308651 (+ 108us) server_negotiation.cc:299] Negotiation successful
0809 12:17:01.308715 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 12:17:01.313061 26832 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:37596:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:17:01.319722 26832 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:17:01.337126 27469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:01.331398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41904 (local address 127.19.183.193:36627)
0809 12:17:01.331969 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:01.331978 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:01.332062 (+ 84us) server_negotiation.cc:407] Connection header received
0809 12:17:01.332277 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:01.332287 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:01.332575 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:01.332748 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:01.335136 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:01.335145 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:01.335158 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:01.335190 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:01.335252 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:01.335278 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:01.335402 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:01.336718 (+ 1316us) server_negotiation.cc:299] Negotiation successful
0809 12:17:01.336826 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":287,"threads_started":1}
I20260809 12:17:01.336496 27468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:01.331167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:36627 (local address 127.0.0.1:41904)
0809 12:17:01.331903 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:01.331956 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:01.332117 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:01.333854 (+ 1737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:01.333866 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:01.333980 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:01.334819 (+ 839us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:17:01.334823 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:01.334865 (+ 42us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:17:01.334948 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:01.335532 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:01.335549 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:01.335557 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 12:17:01.335791 (+ 234us) client_negotiation.cc:240] Negotiation successful
0809 12:17:01.335902 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":345,"threads_started":1}
I20260809 12:17:01.338122 27473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:01.332688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38326 (local address 127.19.183.195:43159)
0809 12:17:01.334815 (+ 2127us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:01.334822 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:01.334862 (+ 40us) server_negotiation.cc:407] Connection header received
0809 12:17:01.334923 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:01.334929 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:01.335104 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:01.335831 (+ 727us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:01.337307 (+ 1476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:01.337314 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:01.337323 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:01.337339 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:01.337387 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:01.337405 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:01.337579 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:01.337664 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 12:17:01.337752 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1947,"thread_start_us":2290,"threads_started":1}
I20260809 12:17:01.338256 27470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:01.332290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:43159 (local address 127.0.0.1:38326)
0809 12:17:01.332877 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:01.332895 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:01.333000 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:01.335298 (+ 2298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:01.335303 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:01.335325 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:01.336138 (+ 813us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:17:01.336141 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:01.336157 (+ 16us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:17:01.336235 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:01.337526 (+ 1291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:01.337532 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:01.337539 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 12:17:01.337687 (+ 148us) client_negotiation.cc:240] Negotiation successful
0809 12:17:01.337744 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":442,"thread_start_us":312,"threads_started":1}
I20260809 12:17:01.338382 27474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:01.333091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:39847 (local address 127.0.0.1:41448)
0809 12:17:01.334123 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:01.334146 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:01.334247 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:01.335298 (+ 1051us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:01.335303 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:01.335325 (+ 22us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:01.336633 (+ 1308us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:17:01.336637 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:01.336652 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:17:01.336711 (+ 59us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:01.337654 (+ 943us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:01.337663 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:01.337668 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 12:17:01.337812 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 12:17:01.337865 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"thread_start_us":743,"threads_started":1}
I20260809 12:17:01.338881 27475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:01.333480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41448 (local address 127.19.183.194:39847)
0809 12:17:01.334787 (+ 1307us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:01.334795 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:01.334825 (+ 30us) server_negotiation.cc:407] Connection header received
0809 12:17:01.334909 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:01.334915 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:01.335105 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:01.335419 (+ 314us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:01.337290 (+ 1871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:01.337309 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:01.337321 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:01.337339 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:01.337394 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:01.337411 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:01.337577 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:01.338376 (+ 799us) server_negotiation.cc:299] Negotiation successful
0809 12:17:01.338479 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1105,"thread_start_us":1355,"threads_started":1}
I20260809 12:17:01.341949 26997 tablet_service.cc:1467] Processing CreateTablet for tablet a704a471213946e48a7b77f0e245f4af (DEFAULT_TABLE table=test-table [id=85640c95c114425aa81a66ebdd138c1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:17:01.342314 27368 tablet_service.cc:1467] Processing CreateTablet for tablet a704a471213946e48a7b77f0e245f4af (DEFAULT_TABLE table=test-table [id=85640c95c114425aa81a66ebdd138c1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:17:01.342315 27183 tablet_service.cc:1467] Processing CreateTablet for tablet a704a471213946e48a7b77f0e245f4af (DEFAULT_TABLE table=test-table [id=85640c95c114425aa81a66ebdd138c1c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:17:01.343273 26997 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a704a471213946e48a7b77f0e245f4af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:01.343717 27183 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a704a471213946e48a7b77f0e245f4af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:01.343717 27368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a704a471213946e48a7b77f0e245f4af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:01.355959 27477 tablet_bootstrap.cc:492] T a704a471213946e48a7b77f0e245f4af P a320908b9b154baf9f3b0f9661265354: Bootstrap starting.
I20260809 12:17:01.358232 27478 tablet_bootstrap.cc:492] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5: Bootstrap starting.
I20260809 12:17:01.359620 27477 tablet_bootstrap.cc:654] T a704a471213946e48a7b77f0e245f4af P a320908b9b154baf9f3b0f9661265354: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:01.359905 27479 tablet_bootstrap.cc:492] T a704a471213946e48a7b77f0e245f4af P f83c4c167b984f42954217da3e34a6b7: Bootstrap starting.
I20260809 12:17:01.360960 27477 log.cc:824] T a704a471213946e48a7b77f0e245f4af P a320908b9b154baf9f3b0f9661265354: Log is configured to *not* fsync() on all Append() calls
I20260809 12:17:01.361682 27478 tablet_bootstrap.cc:654] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:01.363268 27478 log.cc:824] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5: Log is configured to *not* fsync() on all Append() calls
I20260809 12:17:01.363615 27479 tablet_bootstrap.cc:654] T a704a471213946e48a7b77f0e245f4af P f83c4c167b984f42954217da3e34a6b7: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:01.364432 27477 tablet_bootstrap.cc:492] T a704a471213946e48a7b77f0e245f4af P a320908b9b154baf9f3b0f9661265354: No bootstrap required, opened a new log
I20260809 12:17:01.364674 27479 log.cc:824] T a704a471213946e48a7b77f0e245f4af P f83c4c167b984f42954217da3e34a6b7: Log is configured to *not* fsync() on all Append() calls
I20260809 12:17:01.364748 27477 ts_tablet_manager.cc:1397] T a704a471213946e48a7b77f0e245f4af P a320908b9b154baf9f3b0f9661265354: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 12:17:01.367210 27479 tablet_bootstrap.cc:492] T a704a471213946e48a7b77f0e245f4af P f83c4c167b984f42954217da3e34a6b7: No bootstrap required, opened a new log
I20260809 12:17:01.367466 27479 ts_tablet_manager.cc:1397] T a704a471213946e48a7b77f0e245f4af P f83c4c167b984f42954217da3e34a6b7: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.001s
I20260809 12:17:01.369158 27478 tablet_bootstrap.cc:492] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5: No bootstrap required, opened a new log
I20260809 12:17:01.369460 27478 ts_tablet_manager.cc:1397] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.005s
I20260809 12:17:01.374760 27479 raft_consensus.cc:348] T a704a471213946e48a7b77f0e245f4af P f83c4c167b984f42954217da3e34a6b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a320908b9b154baf9f3b0f9661265354" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36627 } } peers { permanent_uuid: "f83c4c167b984f42954217da3e34a6b7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 43159 } } peers { permanent_uuid: "259c08c43afc4a8583b72c96200d17d5" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 39847 } }
I20260809 12:17:01.375133 27479 raft_consensus.cc:374] T a704a471213946e48a7b77f0e245f4af P f83c4c167b984f42954217da3e34a6b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:01.375268 27479 raft_consensus.cc:729] T a704a471213946e48a7b77f0e245f4af P f83c4c167b984f42954217da3e34a6b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f83c4c167b984f42954217da3e34a6b7, State: Initialized, Role: FOLLOWER
I20260809 12:17:01.375448 27477 raft_consensus.cc:348] T a704a471213946e48a7b77f0e245f4af P a320908b9b154baf9f3b0f9661265354 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a320908b9b154baf9f3b0f9661265354" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36627 } } peers { permanent_uuid: "f83c4c167b984f42954217da3e34a6b7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 43159 } } peers { permanent_uuid: "259c08c43afc4a8583b72c96200d17d5" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 39847 } }
I20260809 12:17:01.375864 27477 raft_consensus.cc:374] T a704a471213946e48a7b77f0e245f4af P a320908b9b154baf9f3b0f9661265354 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:01.375963 27477 raft_consensus.cc:729] T a704a471213946e48a7b77f0e245f4af P a320908b9b154baf9f3b0f9661265354 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a320908b9b154baf9f3b0f9661265354, State: Initialized, Role: FOLLOWER
I20260809 12:17:01.375890 27479 consensus_queue.cc:260] T a704a471213946e48a7b77f0e245f4af P f83c4c167b984f42954217da3e34a6b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a320908b9b154baf9f3b0f9661265354" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36627 } } peers { permanent_uuid: "f83c4c167b984f42954217da3e34a6b7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 43159 } } peers { permanent_uuid: "259c08c43afc4a8583b72c96200d17d5" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 39847 } }
I20260809 12:17:01.376637 27477 consensus_queue.cc:260] T a704a471213946e48a7b77f0e245f4af P a320908b9b154baf9f3b0f9661265354 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a320908b9b154baf9f3b0f9661265354" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36627 } } peers { permanent_uuid: "f83c4c167b984f42954217da3e34a6b7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 43159 } } peers { permanent_uuid: "259c08c43afc4a8583b72c96200d17d5" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 39847 } }
I20260809 12:17:01.377018 27453 heartbeater.cc:503] Master 127.19.183.254:41093 was elected leader, sending a full tablet report...
I20260809 12:17:01.378844 27479 ts_tablet_manager.cc:1428] T a704a471213946e48a7b77f0e245f4af P f83c4c167b984f42954217da3e34a6b7: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260809 12:17:01.380853 27478 raft_consensus.cc:348] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a320908b9b154baf9f3b0f9661265354" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36627 } } peers { permanent_uuid: "f83c4c167b984f42954217da3e34a6b7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 43159 } } peers { permanent_uuid: "259c08c43afc4a8583b72c96200d17d5" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 39847 } }
I20260809 12:17:01.381352 27478 raft_consensus.cc:374] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:01.381490 27478 raft_consensus.cc:729] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 259c08c43afc4a8583b72c96200d17d5, State: Initialized, Role: FOLLOWER
I20260809 12:17:01.382200 27478 consensus_queue.cc:260] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a320908b9b154baf9f3b0f9661265354" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36627 } } peers { permanent_uuid: "f83c4c167b984f42954217da3e34a6b7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 43159 } } peers { permanent_uuid: "259c08c43afc4a8583b72c96200d17d5" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 39847 } }
I20260809 12:17:01.383775 27271 heartbeater.cc:503] Master 127.19.183.254:41093 was elected leader, sending a full tablet report...
I20260809 12:17:01.384378 27478 ts_tablet_manager.cc:1428] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260809 12:17:01.385309 27083 heartbeater.cc:503] Master 127.19.183.254:41093 was elected leader, sending a full tablet report...
I20260809 12:17:01.385942 27477 ts_tablet_manager.cc:1428] T a704a471213946e48a7b77f0e245f4af P a320908b9b154baf9f3b0f9661265354: Time spent starting tablet: real 0.021s user 0.015s sys 0.007s
W20260809 12:17:01.394168 27085 tablet.cc:2369] T a704a471213946e48a7b77f0e245f4af P a320908b9b154baf9f3b0f9661265354: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:17:01.407632 27489 raft_consensus.cc:482] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:17:01.407835 27489 raft_consensus.cc:504] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a320908b9b154baf9f3b0f9661265354" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36627 } } peers { permanent_uuid: "f83c4c167b984f42954217da3e34a6b7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 43159 } } peers { permanent_uuid: "259c08c43afc4a8583b72c96200d17d5" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 39847 } }
I20260809 12:17:01.409192 27489 leader_election.cc:288] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a320908b9b154baf9f3b0f9661265354 (127.19.183.193:36627), f83c4c167b984f42954217da3e34a6b7 (127.19.183.195:43159)
I20260809 12:17:01.412686 27469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:01.409486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:45341 (local address 127.19.183.193:36627)
0809 12:17:01.409642 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:01.409646 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:01.410191 (+ 545us) server_negotiation.cc:407] Connection header received
0809 12:17:01.410495 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:01.410500 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:01.410568 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:01.410640 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:01.412046 (+ 1406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:01.412052 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:01.412057 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:01.412082 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:01.412110 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:01.412122 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:01.412243 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:01.412335 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 12:17:01.412399 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 12:17:01.412829 27473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:01.410246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:40703 (local address 127.19.183.195:43159)
0809 12:17:01.410429 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:01.410436 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:01.410886 (+ 450us) server_negotiation.cc:407] Connection header received
0809 12:17:01.411071 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:01.411082 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:01.411134 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:01.411246 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:01.412219 (+ 973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:01.412225 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:01.412229 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:01.412248 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:01.412285 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:01.412296 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:01.412425 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:01.412505 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 12:17:01.412567 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 12:17:01.412920 27490 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:01.409429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:36627 (local address 127.19.183.194:45341)
0809 12:17:01.410079 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:01.410098 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:01.410294 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:01.410889 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:01.410895 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:01.410924 (+ 29us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:01.411814 (+ 890us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:17:01.411818 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:01.411838 (+ 20us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:17:01.411891 (+ 53us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:01.412221 (+ 330us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:01.412227 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:01.412232 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 12:17:01.412387 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 12:17:01.412438 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":335,"threads_started":1}
I20260809 12:17:01.413276 27491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:01.410148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:43159 (local address 127.19.183.194:40703)
0809 12:17:01.410787 (+ 639us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:01.410802 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:01.410927 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:01.411292 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:01.411302 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:01.411329 (+ 27us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:01.412036 (+ 707us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:17:01.412039 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:01.412051 (+ 12us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:17:01.412086 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:01.412397 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:01.412403 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:01.412410 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 12:17:01.412555 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 12:17:01.412618 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":481,"thread_start_us":303,"threads_started":1}
I20260809 12:17:01.413846 27025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a704a471213946e48a7b77f0e245f4af" candidate_uuid: "259c08c43afc4a8583b72c96200d17d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a320908b9b154baf9f3b0f9661265354" is_pre_election: true
I20260809 12:17:01.414052 27391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a704a471213946e48a7b77f0e245f4af" candidate_uuid: "259c08c43afc4a8583b72c96200d17d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f83c4c167b984f42954217da3e34a6b7" is_pre_election: true
I20260809 12:17:01.414322 27025 raft_consensus.cc:2454] T a704a471213946e48a7b77f0e245f4af P a320908b9b154baf9f3b0f9661265354 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 259c08c43afc4a8583b72c96200d17d5 in term 0.
I20260809 12:17:01.414551 27391 raft_consensus.cc:2454] T a704a471213946e48a7b77f0e245f4af P f83c4c167b984f42954217da3e34a6b7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 259c08c43afc4a8583b72c96200d17d5 in term 0.
I20260809 12:17:01.414992 27113 leader_election.cc:302] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 259c08c43afc4a8583b72c96200d17d5, a320908b9b154baf9f3b0f9661265354; no voters:
I20260809 12:17:01.415376 27489 raft_consensus.cc:2781] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:17:01.415518 27489 raft_consensus.cc:482] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:17:01.415639 27489 raft_consensus.cc:3037] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:01.417863 27489 raft_consensus.cc:504] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a320908b9b154baf9f3b0f9661265354" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36627 } } peers { permanent_uuid: "f83c4c167b984f42954217da3e34a6b7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 43159 } } peers { permanent_uuid: "259c08c43afc4a8583b72c96200d17d5" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 39847 } }
I20260809 12:17:01.418424 27489 leader_election.cc:288] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5 [CANDIDATE]: Term 1 election: Requested vote from peers a320908b9b154baf9f3b0f9661265354 (127.19.183.193:36627), f83c4c167b984f42954217da3e34a6b7 (127.19.183.195:43159)
I20260809 12:17:01.418931 27025 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a704a471213946e48a7b77f0e245f4af" candidate_uuid: "259c08c43afc4a8583b72c96200d17d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a320908b9b154baf9f3b0f9661265354"
I20260809 12:17:01.419041 27391 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a704a471213946e48a7b77f0e245f4af" candidate_uuid: "259c08c43afc4a8583b72c96200d17d5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f83c4c167b984f42954217da3e34a6b7"
I20260809 12:17:01.419140 27025 raft_consensus.cc:3037] T a704a471213946e48a7b77f0e245f4af P a320908b9b154baf9f3b0f9661265354 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:01.419243 27391 raft_consensus.cc:3037] T a704a471213946e48a7b77f0e245f4af P f83c4c167b984f42954217da3e34a6b7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:01.422333 27391 raft_consensus.cc:2454] T a704a471213946e48a7b77f0e245f4af P f83c4c167b984f42954217da3e34a6b7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 259c08c43afc4a8583b72c96200d17d5 in term 1.
I20260809 12:17:01.422338 27025 raft_consensus.cc:2454] T a704a471213946e48a7b77f0e245f4af P a320908b9b154baf9f3b0f9661265354 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 259c08c43afc4a8583b72c96200d17d5 in term 1.
I20260809 12:17:01.422753 27113 leader_election.cc:302] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 259c08c43afc4a8583b72c96200d17d5, f83c4c167b984f42954217da3e34a6b7; no voters:
I20260809 12:17:01.423038 27489 raft_consensus.cc:2781] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:17:01.423477 27489 raft_consensus.cc:686] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5 [term 1 LEADER]: Becoming Leader. State: Replica: 259c08c43afc4a8583b72c96200d17d5, State: Running, Role: LEADER
I20260809 12:17:01.423956 27489 consensus_queue.cc:237] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a320908b9b154baf9f3b0f9661265354" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36627 } } peers { permanent_uuid: "f83c4c167b984f42954217da3e34a6b7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 43159 } } peers { permanent_uuid: "259c08c43afc4a8583b72c96200d17d5" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 39847 } }
I20260809 12:17:01.428409 26832 catalog_manager.cc:5495] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5 reported cstate change: term changed from 0 to 1, leader changed from <none> to 259c08c43afc4a8583b72c96200d17d5 (127.19.183.194). New cstate: current_term: 1 leader_uuid: "259c08c43afc4a8583b72c96200d17d5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a320908b9b154baf9f3b0f9661265354" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36627 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f83c4c167b984f42954217da3e34a6b7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 43159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "259c08c43afc4a8583b72c96200d17d5" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 39847 } health_report { overall_health: HEALTHY } } }
I20260809 12:17:01.463429 27475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:01.461528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41458 (local address 127.19.183.194:39847)
0809 12:17:01.461674 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:01.461678 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:01.461694 (+ 16us) server_negotiation.cc:407] Connection header received
0809 12:17:01.461848 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:01.461852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:01.461949 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:01.462069 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:01.462867 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:01.462871 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:01.462874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:01.462888 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:01.462933 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:01.462944 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:01.463039 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:01.463101 (+ 62us) server_negotiation.cc:299] Negotiation successful
0809 12:17:01.463151 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 12:17:01.471303 27391 raft_consensus.cc:1261] T a704a471213946e48a7b77f0e245f4af P f83c4c167b984f42954217da3e34a6b7 [term 1 FOLLOWER]: Refusing update from remote peer 259c08c43afc4a8583b72c96200d17d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:17:01.471303 27025 raft_consensus.cc:1261] T a704a471213946e48a7b77f0e245f4af P a320908b9b154baf9f3b0f9661265354 [term 1 FOLLOWER]: Refusing update from remote peer 259c08c43afc4a8583b72c96200d17d5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:17:01.472256 27492 consensus_queue.cc:1035] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f83c4c167b984f42954217da3e34a6b7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 43159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:17:01.472611 27489 consensus_queue.cc:1035] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a320908b9b154baf9f3b0f9661265354" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 36627 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:17:01.498250 26833 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:37596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:17:01.498518 26833 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:37596:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:17:01.502038 26833 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 707e5cddf8c74405b6fe27760e6ddc6b: Sending DeleteTablet for 3 replicas of tablet a704a471213946e48a7b77f0e245f4af
I20260809 12:17:01.503126 26997 tablet_service.cc:1514] Processing DeleteTablet for tablet a704a471213946e48a7b77f0e245f4af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:17:01 UTC) from {username='slave'} at 127.0.0.1:41904
I20260809 12:17:01.503242 27368 tablet_service.cc:1514] Processing DeleteTablet for tablet a704a471213946e48a7b77f0e245f4af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:17:01 UTC) from {username='slave'} at 127.0.0.1:38326
I20260809 12:17:01.503619 27183 tablet_service.cc:1514] Processing DeleteTablet for tablet a704a471213946e48a7b77f0e245f4af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:17:01 UTC) from {username='slave'} at 127.0.0.1:41448
I20260809 12:17:01.506103 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 26915
I20260809 12:17:01.514182 27518 tablet_replica.cc:330] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5: stopping tablet replica
I20260809 12:17:01.514832 27518 raft_consensus.cc:2229] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:17:01.515357 27513 tablet_replica.cc:330] T a704a471213946e48a7b77f0e245f4af P a320908b9b154baf9f3b0f9661265354: stopping tablet replica
I20260809 12:17:01.515574 27518 raft_consensus.cc:2258] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:17:01.515908 27513 raft_consensus.cc:2229] T a704a471213946e48a7b77f0e245f4af P a320908b9b154baf9f3b0f9661265354 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:17:01.516429 27513 raft_consensus.cc:2258] T a704a471213946e48a7b77f0e245f4af P a320908b9b154baf9f3b0f9661265354 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:17:01.517872 27518 ts_tablet_manager.cc:1905] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:17:01.518661 27513 ts_tablet_manager.cc:1905] T a704a471213946e48a7b77f0e245f4af P a320908b9b154baf9f3b0f9661265354: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:17:01.525317 27518 ts_tablet_manager.cc:1918] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:17:01.525712 27513 ts_tablet_manager.cc:1918] T a704a471213946e48a7b77f0e245f4af P a320908b9b154baf9f3b0f9661265354: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:17:01.525930 27516 tablet_replica.cc:330] T a704a471213946e48a7b77f0e245f4af P f83c4c167b984f42954217da3e34a6b7: stopping tablet replica
I20260809 12:17:01.525941 27513 log.cc:1196] T a704a471213946e48a7b77f0e245f4af P a320908b9b154baf9f3b0f9661265354: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/a704a471213946e48a7b77f0e245f4af
I20260809 12:17:01.526443 27516 raft_consensus.cc:2229] T a704a471213946e48a7b77f0e245f4af P f83c4c167b984f42954217da3e34a6b7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:17:01.526506 27513 ts_tablet_manager.cc:1939] T a704a471213946e48a7b77f0e245f4af P a320908b9b154baf9f3b0f9661265354: Deleting consensus metadata
I20260809 12:17:01.527438 27516 raft_consensus.cc:2258] T a704a471213946e48a7b77f0e245f4af P f83c4c167b984f42954217da3e34a6b7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:17:01.528153 27469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:01.508135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41916 (local address 127.19.183.193:36627)
0809 12:17:01.508360 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:01.508368 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:01.525286 (+ 16918us) server_negotiation.cc:407] Connection header received
0809 12:17:01.525845 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:01.525851 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:01.525965 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:01.526079 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:01.527280 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:01.527286 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:01.527291 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:01.527313 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:01.527354 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:01.527370 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:01.527466 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:01.527730 (+ 264us) server_negotiation.cc:299] Negotiation successful
0809 12:17:01.527795 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 12:17:01.528513 26811 catalog_manager.cc:4841] TS a320908b9b154baf9f3b0f9661265354 (127.19.183.193:36627): tablet a704a471213946e48a7b77f0e245f4af (table test-table [id=85640c95c114425aa81a66ebdd138c1c]) successfully deleted
I20260809 12:17:01.529078 27075 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:01.529724 27516 ts_tablet_manager.cc:1905] T a704a471213946e48a7b77f0e245f4af P f83c4c167b984f42954217da3e34a6b7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:17:01.537818 27516 ts_tablet_manager.cc:1918] T a704a471213946e48a7b77f0e245f4af P f83c4c167b984f42954217da3e34a6b7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:17:01.537969 27516 log.cc:1196] T a704a471213946e48a7b77f0e245f4af P f83c4c167b984f42954217da3e34a6b7: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/a704a471213946e48a7b77f0e245f4af
I20260809 12:17:01.538520 27516 ts_tablet_manager.cc:1939] T a704a471213946e48a7b77f0e245f4af P f83c4c167b984f42954217da3e34a6b7: Deleting consensus metadata
I20260809 12:17:01.540376 27518 log.cc:1196] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/a704a471213946e48a7b77f0e245f4af
I20260809 12:17:01.540498 26811 catalog_manager.cc:4841] TS f83c4c167b984f42954217da3e34a6b7 (127.19.183.195:43159): tablet a704a471213946e48a7b77f0e245f4af (table test-table [id=85640c95c114425aa81a66ebdd138c1c]) successfully deleted
I20260809 12:17:01.541101 27518 ts_tablet_manager.cc:1939] T a704a471213946e48a7b77f0e245f4af P 259c08c43afc4a8583b72c96200d17d5: Deleting consensus metadata
I20260809 12:17:01.542930 26811 catalog_manager.cc:4841] TS 259c08c43afc4a8583b72c96200d17d5 (127.19.183.194:39847): tablet a704a471213946e48a7b77f0e245f4af (table test-table [id=85640c95c114425aa81a66ebdd138c1c]) successfully deleted
I20260809 12:17:01.621447 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 26915
I20260809 12:17:01.650929 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 27091
I20260809 12:17:01.654156 27475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:01.651732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41464 (local address 127.19.183.194:39847)
0809 12:17:01.651897 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:01.651901 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:01.651923 (+ 22us) server_negotiation.cc:407] Connection header received
0809 12:17:01.652204 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:01.652208 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:01.652266 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:01.652350 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:01.653427 (+ 1077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:01.653431 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:01.653434 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:01.653450 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:01.653482 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:01.653495 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:01.653616 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:01.653758 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 12:17:01.653810 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 12:17:01.654906 27266 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:01.765995 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 27091
I20260809 12:17:01.793905 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 27276
I20260809 12:17:01.796653 27473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:01.794657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38342 (local address 127.19.183.195:43159)
0809 12:17:01.794881 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:01.794885 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:01.794934 (+ 49us) server_negotiation.cc:407] Connection header received
0809 12:17:01.795087 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:01.795108 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:01.795187 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:01.795269 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:01.796050 (+ 781us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:01.796054 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:01.796057 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:01.796072 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:01.796103 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:01.796114 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:01.796225 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:01.796288 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 12:17:01.796331 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 12:17:01.797372 27448 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:01.883101 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 27276
I20260809 12:17:01.907907 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 26786
I20260809 12:17:01.909085 26876 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:02.014542 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 26786
2026-08-09T12:17:02Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (2682 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 12:17:02.068338 20191 mini_oidc.cc:152] Starting JWKS server
I20260809 12:17:02.071264 20191 webserver.cc:460] Webserver started at https://127.0.0.1:46423/ using document root <none> and password file <none>
I20260809 12:17:02.071717 20191 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 12:17:02.072410 20191 webserver.cc:460] Webserver started at http://127.0.0.1:34993/ using document root <none> and password file <none>
2026-08-09T12:17:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:17:02Z Disabled control of system clock
I20260809 12:17:02.106256 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:41943
--webserver_interface=127.19.183.254
--webserver_port=0
--builtin_ntp_servers=127.19.183.212:33345
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:41943
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46423/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786277774779245-20191-0/testchainca.cert with env {}
W20260809 12:17:02.430215 27534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:02.430549 27534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:02.430758 27534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:02.437919 27534 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:17:02.438061 27534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:02.438153 27534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:02.438256 27534 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:17:02.438341 27534 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 12:17:02.438431 27534 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 12:17:02.438491 27534 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46423/jwks.json
I20260809 12:17:02.447584 27534 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:33345
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:41943
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:41943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46423/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786277774779245-20191-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:02.448846 27534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:02.450132 27534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:02.461725 27539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:02.461881 27541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:02.461817 27546 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:02.463022 27534 server_base.cc:1034] running on GCE node
I20260809 12:17:02.463744 27534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:02.464972 27534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:02.466212 27534 hybrid_clock.cc:648] HybridClock initialized: now 1786277822466178 us; error 47 us; skew 500 ppm
I20260809 12:17:02.466526 27534 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:17:02.468664 27534 webserver.cc:460] Webserver started at http://127.19.183.254:46429/ using document root <none> and password file <none>
I20260809 12:17:02.469369 27534 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:02.469509 27534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:02.469830 27534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:02.472839 27534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "5c922e42b7f2438dbb21d4912e869d21"
format_stamp: "Formatted at 2026-08-09 12:17:02 on dist-test-slave-39l8"
server_key: "5bf727b53a4da8a0a915f0fd53d5349d"
server_key_iv: "5ea285a7ef1613a8bc92d775ec7643b3"
server_key_version: "encryptionkey@0"
I20260809 12:17:02.473749 27534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "5c922e42b7f2438dbb21d4912e869d21"
format_stamp: "Formatted at 2026-08-09 12:17:02 on dist-test-slave-39l8"
server_key: "5bf727b53a4da8a0a915f0fd53d5349d"
server_key_iv: "5ea285a7ef1613a8bc92d775ec7643b3"
server_key_version: "encryptionkey@0"
I20260809 12:17:02.478936 27534 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260809 12:17:02.482066 27553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:02.483346 27534 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:17:02.483530 27534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "5c922e42b7f2438dbb21d4912e869d21"
format_stamp: "Formatted at 2026-08-09 12:17:02 on dist-test-slave-39l8"
server_key: "5bf727b53a4da8a0a915f0fd53d5349d"
server_key_iv: "5ea285a7ef1613a8bc92d775ec7643b3"
server_key_version: "encryptionkey@0"
I20260809 12:17:02.483734 27534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:02.507819 27534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:02.508590 27534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:02.508941 27534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:02.527271 27534 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:41943
I20260809 12:17:02.527313 27627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:41943 every 8 connection(s)
I20260809 12:17:02.528527 27534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:17:02.532049 27628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:02.536096 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 27534
I20260809 12:17:02.536317 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:17:02.537108 20191 external_mini_cluster.cc:1428] Setting key 71dd0d9f1067828a833fdad779ff1eb7
I20260809 12:17:02.543125 27628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c922e42b7f2438dbb21d4912e869d21: Bootstrap starting.
I20260809 12:17:02.545958 27628 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c922e42b7f2438dbb21d4912e869d21: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:02.547024 27628 log.cc:824] T 00000000000000000000000000000000 P 5c922e42b7f2438dbb21d4912e869d21: Log is configured to *not* fsync() on all Append() calls
I20260809 12:17:02.550151 27628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c922e42b7f2438dbb21d4912e869d21: No bootstrap required, opened a new log
I20260809 12:17:02.550478 27631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:02.539097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50362 (local address 127.19.183.254:41943)
0809 12:17:02.540022 (+ 925us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:02.540032 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:02.540072 (+ 40us) server_negotiation.cc:407] Connection header received
0809 12:17:02.541897 (+ 1825us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:02.541919 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:02.542326 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:02.542678 (+ 352us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:02.543229 (+ 551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:02.544454 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:02.545764 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:02.546467 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:02.546676 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:02.547825 (+ 1149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:02.547831 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:02.547844 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:02.548472 (+ 628us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:02.548564 (+ 92us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:02.548589 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:02.548731 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:02.548960 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 12:17:02.549263 (+ 303us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":331,"threads_started":1}
I20260809 12:17:02.554152 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:17:02.558662 27628 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c922e42b7f2438dbb21d4912e869d21 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c922e42b7f2438dbb21d4912e869d21" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41943 } }
I20260809 12:17:02.559064 27628 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c922e42b7f2438dbb21d4912e869d21 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:02.559147 27628 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c922e42b7f2438dbb21d4912e869d21 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c922e42b7f2438dbb21d4912e869d21, State: Initialized, Role: FOLLOWER
I20260809 12:17:02.559656 27628 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c922e42b7f2438dbb21d4912e869d21 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c922e42b7f2438dbb21d4912e869d21" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41943 } }
I20260809 12:17:02.559854 27628 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c922e42b7f2438dbb21d4912e869d21 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:17:02.559947 27628 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c922e42b7f2438dbb21d4912e869d21 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:17:02.560079 27628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c922e42b7f2438dbb21d4912e869d21 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:02.562027 27628 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c922e42b7f2438dbb21d4912e869d21 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c922e42b7f2438dbb21d4912e869d21" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41943 } }
I20260809 12:17:02.562500 27628 leader_election.cc:302] T 00000000000000000000000000000000 P 5c922e42b7f2438dbb21d4912e869d21 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c922e42b7f2438dbb21d4912e869d21; no voters:
I20260809 12:17:02.563040 27628 leader_election.cc:288] T 00000000000000000000000000000000 P 5c922e42b7f2438dbb21d4912e869d21 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:17:02.563205 27634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c922e42b7f2438dbb21d4912e869d21 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:17:02.563941 27634 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c922e42b7f2438dbb21d4912e869d21 [term 1 LEADER]: Becoming Leader. State: Replica: 5c922e42b7f2438dbb21d4912e869d21, State: Running, Role: LEADER
I20260809 12:17:02.564397 27634 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c922e42b7f2438dbb21d4912e869d21 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c922e42b7f2438dbb21d4912e869d21" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41943 } }
I20260809 12:17:02.565289 27628 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c922e42b7f2438dbb21d4912e869d21 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:17:02.567672 27636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c922e42b7f2438dbb21d4912e869d21 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c922e42b7f2438dbb21d4912e869d21. Latest consensus state: current_term: 1 leader_uuid: "5c922e42b7f2438dbb21d4912e869d21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c922e42b7f2438dbb21d4912e869d21" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41943 } } }
I20260809 12:17:02.568130 27636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c922e42b7f2438dbb21d4912e869d21 [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:02.567636 27635 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c922e42b7f2438dbb21d4912e869d21 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c922e42b7f2438dbb21d4912e869d21" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c922e42b7f2438dbb21d4912e869d21" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41943 } } }
I20260809 12:17:02.568627 27635 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c922e42b7f2438dbb21d4912e869d21 [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:02.571336 27643 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:17:02.576449 27643 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:17:02.585305 27643 catalog_manager.cc:1347] Generated new cluster ID: 34cfac1602cd45c68e8d483b06d5c5b9
I20260809 12:17:02.585431 27643 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:17:02.595155 27643 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:17:02.596760 27643 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:17:02.606057 27643 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c922e42b7f2438dbb21d4912e869d21: Generated new TSK 0
I20260809 12:17:02.606921 27643 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 12:17:02.650359 27672 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 12:17:02.650589 27672 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 12:17:02.651436 27631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:02.630929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50364 (local address 127.19.183.254:41943)
0809 12:17:02.631172 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:02.631182 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:02.631653 (+ 471us) server_negotiation.cc:407] Connection header received
0809 12:17:02.632075 (+ 422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:02.632079 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:02.632153 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:02.632268 (+ 115us) server_negotiation.cc:226] Negotiated authn=JWT
0809 12:17:02.632930 (+ 662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:02.633599 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:02.635648 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:02.636052 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:02.636140 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:02.636754 (+ 614us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 12:17:02.650992 (+ 14238us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46423/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 12:17:02.651236 (+ 244us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:50364: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46423/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":62}
W20260809 12:17:02.651582 27631 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:50364: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46423/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 12:17:02.653618 27670 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:02.630841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:41943 (local address 127.0.0.1:50364)
0809 12:17:02.631536 (+ 695us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:02.631557 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:02.631797 (+ 240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:02.632335 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:02.632341 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:02.632355 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0809 12:17:02.632774 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:02.632789 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:02.633752 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:02.633758 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:02.635513 (+ 1755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:02.635527 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:02.636240 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:02.636247 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:02.636440 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:02.636521 (+ 81us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 12:17:02.651248 (+ 14727us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46423/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 12:17:02.651570 (+ 322us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.183.254:41943: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46423/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":366,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.19.183.254:41943: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46423/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 12:17:02.655551 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 27534
I20260809 12:17:02.656719 27618 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:02.752556 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 27534
2026-08-09T12:17:02Z chronyd exiting
I20260809 12:17:02.812613 20191 test_util.cc:175] -----------------------------------------------
I20260809 12:17:02.812774 20191 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786277774779245-20191-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (749 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 12:17:02.816628 20191 mini_oidc.cc:152] Starting JWKS server
I20260809 12:17:02.818403 20191 webserver.cc:460] Webserver started at https://127.0.0.1:45829/ using document root <none> and password file <none>
I20260809 12:17:02.818655 20191 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 12:17:02.819315 20191 webserver.cc:460] Webserver started at http://127.0.0.1:32879/ using document root <none> and password file <none>
2026-08-09T12:17:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:17:02Z Disabled control of system clock
I20260809 12:17:02.854251 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:38505
--webserver_interface=127.19.183.254
--webserver_port=0
--builtin_ntp_servers=127.19.183.212:37153
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:38505
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45829/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786277774779245-20191-0/testchainca.cert with env {}
W20260809 12:17:03.176453 27681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:03.176774 27681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:03.176921 27681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:03.183965 27681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:17:03.184110 27681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:03.184204 27681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:03.184296 27681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:17:03.184365 27681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 12:17:03.184453 27681 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 12:17:03.184530 27681 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:45829/jwks.json
I20260809 12:17:03.194380 27681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:37153
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:38505
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:38505
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:45829/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786277774779245-20191-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:03.195588 27681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:03.196892 27681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:03.209936 27692 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:03.209898 27690 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:03.209972 27688 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:03.210741 27681 server_base.cc:1034] running on GCE node
I20260809 12:17:03.211450 27681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:03.212746 27681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:03.213948 27681 hybrid_clock.cc:648] HybridClock initialized: now 1786277823213906 us; error 34 us; skew 500 ppm
I20260809 12:17:03.214320 27681 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:17:03.219636 27681 webserver.cc:460] Webserver started at http://127.19.183.254:44235/ using document root <none> and password file <none>
I20260809 12:17:03.220346 27681 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:03.220474 27681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:03.220824 27681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:03.223775 27681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "fb1ca82612274389becdc6a00c780339"
format_stamp: "Formatted at 2026-08-09 12:17:03 on dist-test-slave-39l8"
server_key: "e7dbf864661b40737162c1ca178e76fe"
server_key_iv: "f937cc67a5dc222b6233f60f368bccb9"
server_key_version: "encryptionkey@0"
I20260809 12:17:03.224538 27681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "fb1ca82612274389becdc6a00c780339"
format_stamp: "Formatted at 2026-08-09 12:17:03 on dist-test-slave-39l8"
server_key: "e7dbf864661b40737162c1ca178e76fe"
server_key_iv: "f937cc67a5dc222b6233f60f368bccb9"
server_key_version: "encryptionkey@0"
I20260809 12:17:03.229570 27681 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:17:03.233083 27699 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:03.234349 27681 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:17:03.234526 27681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "fb1ca82612274389becdc6a00c780339"
format_stamp: "Formatted at 2026-08-09 12:17:03 on dist-test-slave-39l8"
server_key: "e7dbf864661b40737162c1ca178e76fe"
server_key_iv: "f937cc67a5dc222b6233f60f368bccb9"
server_key_version: "encryptionkey@0"
I20260809 12:17:03.234782 27681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:03.253198 27681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:03.254052 27681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:03.254381 27681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:03.272742 27681 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:38505
I20260809 12:17:03.272773 27782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:38505 every 8 connection(s)
I20260809 12:17:03.273988 27681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:17:03.277447 27785 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:03.284363 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 27681
I20260809 12:17:03.284556 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:17:03.285329 20191 external_mini_cluster.cc:1428] Setting key cdf1d24e4c316a595b48ebe03da45cd4
I20260809 12:17:03.287866 27785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb1ca82612274389becdc6a00c780339: Bootstrap starting.
I20260809 12:17:03.291875 27785 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb1ca82612274389becdc6a00c780339: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:03.293226 27785 log.cc:824] T 00000000000000000000000000000000 P fb1ca82612274389becdc6a00c780339: Log is configured to *not* fsync() on all Append() calls
I20260809 12:17:03.296239 27785 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb1ca82612274389becdc6a00c780339: No bootstrap required, opened a new log
I20260809 12:17:03.300688 27791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:03.287296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49408 (local address 127.19.183.254:38505)
0809 12:17:03.288322 (+ 1026us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:03.288341 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:03.288391 (+ 50us) server_negotiation.cc:407] Connection header received
0809 12:17:03.290876 (+ 2485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:03.290905 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:03.291453 (+ 548us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:03.291879 (+ 426us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:03.292465 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:03.294290 (+ 1825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:03.295643 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:03.296660 (+ 1017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:03.296851 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:03.297890 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:03.297900 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:03.297918 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:03.298638 (+ 720us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:03.298753 (+ 115us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:03.298783 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:03.298927 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:03.299143 (+ 216us) server_negotiation.cc:299] Negotiation successful
0809 12:17:03.299461 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":533,"thread_start_us":301,"threads_started":1}
I20260809 12:17:03.304380 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:17:03.305697 27785 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb1ca82612274389becdc6a00c780339 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb1ca82612274389becdc6a00c780339" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38505 } }
I20260809 12:17:03.306041 27785 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb1ca82612274389becdc6a00c780339 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:03.306138 27785 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb1ca82612274389becdc6a00c780339 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb1ca82612274389becdc6a00c780339, State: Initialized, Role: FOLLOWER
I20260809 12:17:03.306842 27785 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb1ca82612274389becdc6a00c780339 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb1ca82612274389becdc6a00c780339" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38505 } }
I20260809 12:17:03.307102 27785 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb1ca82612274389becdc6a00c780339 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:17:03.307260 27785 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb1ca82612274389becdc6a00c780339 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:17:03.307433 27785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb1ca82612274389becdc6a00c780339 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:03.309527 27785 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb1ca82612274389becdc6a00c780339 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb1ca82612274389becdc6a00c780339" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38505 } }
I20260809 12:17:03.310016 27785 leader_election.cc:302] T 00000000000000000000000000000000 P fb1ca82612274389becdc6a00c780339 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb1ca82612274389becdc6a00c780339; no voters:
I20260809 12:17:03.310537 27785 leader_election.cc:288] T 00000000000000000000000000000000 P fb1ca82612274389becdc6a00c780339 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:17:03.310690 27793 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb1ca82612274389becdc6a00c780339 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:17:03.311455 27793 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb1ca82612274389becdc6a00c780339 [term 1 LEADER]: Becoming Leader. State: Replica: fb1ca82612274389becdc6a00c780339, State: Running, Role: LEADER
I20260809 12:17:03.311980 27793 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb1ca82612274389becdc6a00c780339 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb1ca82612274389becdc6a00c780339" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38505 } }
I20260809 12:17:03.312675 27785 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb1ca82612274389becdc6a00c780339 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:17:03.314862 27795 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb1ca82612274389becdc6a00c780339 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb1ca82612274389becdc6a00c780339. Latest consensus state: current_term: 1 leader_uuid: "fb1ca82612274389becdc6a00c780339" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb1ca82612274389becdc6a00c780339" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38505 } } }
I20260809 12:17:03.315394 27795 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb1ca82612274389becdc6a00c780339 [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:03.315181 27794 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb1ca82612274389becdc6a00c780339 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb1ca82612274389becdc6a00c780339" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb1ca82612274389becdc6a00c780339" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38505 } } }
I20260809 12:17:03.315582 27794 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb1ca82612274389becdc6a00c780339 [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:03.317646 27801 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:17:03.322810 27801 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:17:03.331431 27801 catalog_manager.cc:1347] Generated new cluster ID: 6d21dc0c0860477abc68b42ef09508c1
I20260809 12:17:03.331537 27801 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:17:03.343585 27801 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:17:03.345314 27801 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:17:03.364178 27801 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb1ca82612274389becdc6a00c780339: Generated new TSK 0
I20260809 12:17:03.365253 27801 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 12:17:03.399049 27822 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 12:17:03.399185 27822 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 12:17:03.400352 27791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:03.380893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49418 (local address 127.19.183.254:38505)
0809 12:17:03.381037 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:03.381041 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:03.381456 (+ 415us) server_negotiation.cc:407] Connection header received
0809 12:17:03.381699 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:03.381704 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:03.381800 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:03.381954 (+ 154us) server_negotiation.cc:226] Negotiated authn=JWT
0809 12:17:03.382407 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:03.383614 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:03.384954 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:03.385666 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:03.385798 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:03.386223 (+ 425us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 12:17:03.399701 (+ 13478us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45829/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 12:17:03.400067 (+ 366us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:49418: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45829/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":37}
W20260809 12:17:03.400370 27821 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:03.380758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:38505 (local address 127.0.0.1:49418)
0809 12:17:03.381348 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:03.381368 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:03.381481 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:03.381982 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:03.381985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:03.381995 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0809 12:17:03.382238 (+ 243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:03.382246 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:03.383770 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:03.383774 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:03.384813 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:03.384824 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:03.385811 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:03.385815 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:03.385940 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:03.385969 (+ 29us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 12:17:03.399896 (+ 13927us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45829/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 12:17:03.400084 (+ 188us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.183.254:38505: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45829/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":328,"threads_started":1}
W20260809 12:17:03.400473 27791 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:49418: Failed to load JWKS: Error downloading JWKS from 'https://localhost:45829/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260809 12:17:03.401556 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 27681
I20260809 12:17:03.402850 27774 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:03.493818 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 27681
2026-08-09T12:17:03Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (738 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 12:17:03.555610 20191 mini_oidc.cc:152] Starting JWKS server
I20260809 12:17:03.557485 20191 webserver.cc:460] Webserver started at https://127.0.0.1:36705/ using document root <none> and password file <none>
I20260809 12:17:03.557785 20191 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 12:17:03.558405 20191 webserver.cc:460] Webserver started at http://127.0.0.1:46117/ using document root <none> and password file <none>
2026-08-09T12:17:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:17:03Z Disabled control of system clock
I20260809 12:17:03.592638 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:45375
--webserver_interface=127.19.183.254
--webserver_port=0
--builtin_ntp_servers=127.19.183.212:34119
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:45375
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36705/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 12:17:03.930226 27835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:03.930540 27835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:03.930694 27835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:03.937529 27835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:17:03.937637 27835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:03.937674 27835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:03.937722 27835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:17:03.937757 27835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 12:17:03.937817 27835 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 12:17:03.937855 27835 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36705/jwks.json
I20260809 12:17:03.946471 27835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:34119
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:45375
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:45375
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36705/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:03.947765 27835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:03.949451 27835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:03.960987 27845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:03.961066 27842 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:03.961580 27835 server_base.cc:1034] running on GCE node
W20260809 12:17:03.961700 27841 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:03.962533 27835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:03.963838 27835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:03.965040 27835 hybrid_clock.cc:648] HybridClock initialized: now 1786277823964995 us; error 39 us; skew 500 ppm
I20260809 12:17:03.965332 27835 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:17:03.967542 27835 webserver.cc:460] Webserver started at http://127.19.183.254:40073/ using document root <none> and password file <none>
I20260809 12:17:03.968181 27835 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:03.968267 27835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:03.968523 27835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:03.971266 27835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "c3e9fc3894824b7fa50568d2b19a7335"
format_stamp: "Formatted at 2026-08-09 12:17:03 on dist-test-slave-39l8"
server_key: "72d3c818ee71da4acf5784bf240e6841"
server_key_iv: "f774da0cd68db50911d9ddd63c631260"
server_key_version: "encryptionkey@0"
I20260809 12:17:03.972071 27835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "c3e9fc3894824b7fa50568d2b19a7335"
format_stamp: "Formatted at 2026-08-09 12:17:03 on dist-test-slave-39l8"
server_key: "72d3c818ee71da4acf5784bf240e6841"
server_key_iv: "f774da0cd68db50911d9ddd63c631260"
server_key_version: "encryptionkey@0"
I20260809 12:17:03.976771 27835 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260809 12:17:03.980077 27853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:03.981323 27835 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:17:03.981457 27835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "c3e9fc3894824b7fa50568d2b19a7335"
format_stamp: "Formatted at 2026-08-09 12:17:03 on dist-test-slave-39l8"
server_key: "72d3c818ee71da4acf5784bf240e6841"
server_key_iv: "f774da0cd68db50911d9ddd63c631260"
server_key_version: "encryptionkey@0"
I20260809 12:17:03.981611 27835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:03.997951 27835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:03.998624 27835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:03.998991 27835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:04.021960 27835 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:45375
I20260809 12:17:04.022007 27930 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:45375 every 8 connection(s)
I20260809 12:17:04.023488 27835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:17:04.027503 27931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:04.033329 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 27835
I20260809 12:17:04.033550 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:17:04.034227 20191 external_mini_cluster.cc:1428] Setting key 58f9e232c45bf060e57dae950e24426b
I20260809 12:17:04.037961 27931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3e9fc3894824b7fa50568d2b19a7335: Bootstrap starting.
I20260809 12:17:04.041251 27931 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3e9fc3894824b7fa50568d2b19a7335: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:04.042526 27931 log.cc:824] T 00000000000000000000000000000000 P c3e9fc3894824b7fa50568d2b19a7335: Log is configured to *not* fsync() on all Append() calls
I20260809 12:17:04.045686 27931 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3e9fc3894824b7fa50568d2b19a7335: No bootstrap required, opened a new log
I20260809 12:17:04.047870 27936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:04.036162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53642 (local address 127.19.183.254:45375)
0809 12:17:04.037109 (+ 947us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:04.037120 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:04.037161 (+ 41us) server_negotiation.cc:407] Connection header received
0809 12:17:04.038913 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:04.038934 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:04.039369 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:04.039677 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:04.040287 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:04.041589 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:04.043482 (+ 1893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:04.044259 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:04.044384 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:04.045595 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:04.045602 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:04.045614 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:04.046213 (+ 599us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:04.046281 (+ 68us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:04.046302 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:04.046391 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:04.046618 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 12:17:04.046976 (+ 358us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":336,"threads_started":1}
I20260809 12:17:04.051568 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:17:04.053562 27931 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3e9fc3894824b7fa50568d2b19a7335 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3e9fc3894824b7fa50568d2b19a7335" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45375 } }
I20260809 12:17:04.053974 27931 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3e9fc3894824b7fa50568d2b19a7335 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:04.054112 27931 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3e9fc3894824b7fa50568d2b19a7335 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3e9fc3894824b7fa50568d2b19a7335, State: Initialized, Role: FOLLOWER
I20260809 12:17:04.054677 27931 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3e9fc3894824b7fa50568d2b19a7335 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3e9fc3894824b7fa50568d2b19a7335" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45375 } }
I20260809 12:17:04.054975 27931 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3e9fc3894824b7fa50568d2b19a7335 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:17:04.055114 27931 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3e9fc3894824b7fa50568d2b19a7335 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:17:04.055240 27931 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3e9fc3894824b7fa50568d2b19a7335 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:04.057196 27931 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3e9fc3894824b7fa50568d2b19a7335 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3e9fc3894824b7fa50568d2b19a7335" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45375 } }
I20260809 12:17:04.057626 27931 leader_election.cc:302] T 00000000000000000000000000000000 P c3e9fc3894824b7fa50568d2b19a7335 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3e9fc3894824b7fa50568d2b19a7335; no voters:
I20260809 12:17:04.058202 27931 leader_election.cc:288] T 00000000000000000000000000000000 P c3e9fc3894824b7fa50568d2b19a7335 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:17:04.058346 27938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3e9fc3894824b7fa50568d2b19a7335 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:17:04.059571 27938 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3e9fc3894824b7fa50568d2b19a7335 [term 1 LEADER]: Becoming Leader. State: Replica: c3e9fc3894824b7fa50568d2b19a7335, State: Running, Role: LEADER
I20260809 12:17:04.060073 27938 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3e9fc3894824b7fa50568d2b19a7335 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3e9fc3894824b7fa50568d2b19a7335" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45375 } }
I20260809 12:17:04.060371 27931 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3e9fc3894824b7fa50568d2b19a7335 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:17:04.063254 27939 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3e9fc3894824b7fa50568d2b19a7335 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3e9fc3894824b7fa50568d2b19a7335" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3e9fc3894824b7fa50568d2b19a7335" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45375 } } }
I20260809 12:17:04.063971 27939 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3e9fc3894824b7fa50568d2b19a7335 [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:04.063280 27941 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3e9fc3894824b7fa50568d2b19a7335 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3e9fc3894824b7fa50568d2b19a7335. Latest consensus state: current_term: 1 leader_uuid: "c3e9fc3894824b7fa50568d2b19a7335" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3e9fc3894824b7fa50568d2b19a7335" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45375 } } }
I20260809 12:17:04.069245 27941 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3e9fc3894824b7fa50568d2b19a7335 [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:04.070035 27951 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:17:04.075304 27951 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:17:04.084192 27951 catalog_manager.cc:1347] Generated new cluster ID: eebe9d548f65435dadeeed778a34a6d8
I20260809 12:17:04.084348 27951 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:17:04.096011 27951 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:17:04.097224 27951 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:17:04.108330 27951 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3e9fc3894824b7fa50568d2b19a7335: Generated new TSK 0
I20260809 12:17:04.109160 27951 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:17:04.132460 27936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:04.123798 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53644 (local address 127.19.183.254:45375)
0809 12:17:04.124056 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:04.124066 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:04.124599 (+ 533us) server_negotiation.cc:407] Connection header received
0809 12:17:04.124855 (+ 256us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:04.124863 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:04.124932 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:04.125049 (+ 117us) server_negotiation.cc:226] Negotiated authn=JWT
0809 12:17:04.125643 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:04.126343 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:04.128267 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:04.128695 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:04.128783 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:04.129444 (+ 661us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 12:17:04.131806 (+ 2362us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36705/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 12:17:04.132151 (+ 345us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53644: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36705/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 12:17:04.132594 27936 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53644: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36705/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 12:17:04.134618 27971 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:04.123799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45375 (local address 127.0.0.1:53644)
0809 12:17:04.124486 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:04.124509 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:04.124667 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:04.125106 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:04.125113 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:04.125127 (+ 14us) client_negotiation.cc:189] Negotiated authn=JWT
0809 12:17:04.125477 (+ 350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:04.125490 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:04.126481 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:04.126489 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:04.128128 (+ 1639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:04.128143 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:04.128912 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:04.128925 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:04.129130 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:04.129176 (+ 46us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 12:17:04.132034 (+ 2858us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36705/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 12:17:04.134183 (+ 2149us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.183.254:45375: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36705/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":552,"mutex_wait_us":1824,"thread_start_us":384,"threads_started":1}
I20260809 12:17:04.139097 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 27835
I20260809 12:17:04.140156 27924 generic_service.cc:196] Checking for leaks (request via RPC)
E20260809 12:17:04.146447 27973 webserver.cc:533] Webserver: sslize failed:
E20260809 12:17:04.146567 27973 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 12:17:04.229915 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 27835
2026-08-09T12:17:04Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (739 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T12:17:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:17:04Z Disabled control of system clock
I20260809 12:17:04.328742 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:41541
--webserver_interface=127.19.183.254
--webserver_port=0
--builtin_ntp_servers=127.19.183.212:44805
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:41541
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786277774779245-20191-0/testchainca.cert with env {}
W20260809 12:17:04.655728 27982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:04.656018 27982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:04.656204 27982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:04.662995 27982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:17:04.663120 27982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:04.663187 27982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:04.663306 27982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:17:04.663394 27982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 12:17:04.663477 27982 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 12:17:04.663527 27982 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 12:17:04.672590 27982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:44805
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:41541
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:41541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786277774779245-20191-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:04.673764 27982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:04.674997 27982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:04.686535 27995 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:04.686583 27989 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:04.686483 27990 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:04.687613 27982 server_base.cc:1034] running on GCE node
I20260809 12:17:04.688311 27982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:04.689783 27982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:04.691005 27982 hybrid_clock.cc:648] HybridClock initialized: now 1786277824690971 us; error 47 us; skew 500 ppm
I20260809 12:17:04.691347 27982 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:17:04.693532 27982 webserver.cc:460] Webserver started at http://127.19.183.254:39107/ using document root <none> and password file <none>
I20260809 12:17:04.694203 27982 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:04.694339 27982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:04.694643 27982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:04.697396 27982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "60c16d917e3d4d2b962ee476f393365c"
format_stamp: "Formatted at 2026-08-09 12:17:04 on dist-test-slave-39l8"
server_key: "3cd1241cdb10163e2074806ea51e7e7b"
server_key_iv: "dc357f8325388137eba4f5f7362dea4f"
server_key_version: "encryptionkey@0"
I20260809 12:17:04.698187 27982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "60c16d917e3d4d2b962ee476f393365c"
format_stamp: "Formatted at 2026-08-09 12:17:04 on dist-test-slave-39l8"
server_key: "3cd1241cdb10163e2074806ea51e7e7b"
server_key_iv: "dc357f8325388137eba4f5f7362dea4f"
server_key_version: "encryptionkey@0"
I20260809 12:17:04.703133 27982 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 12:17:04.706302 28007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:04.707590 27982 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:17:04.707763 27982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "60c16d917e3d4d2b962ee476f393365c"
format_stamp: "Formatted at 2026-08-09 12:17:04 on dist-test-slave-39l8"
server_key: "3cd1241cdb10163e2074806ea51e7e7b"
server_key_iv: "dc357f8325388137eba4f5f7362dea4f"
server_key_version: "encryptionkey@0"
I20260809 12:17:04.707967 27982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:04.722185 27982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:04.723039 27982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:04.723310 27982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:04.741221 27982 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:41541
I20260809 12:17:04.741251 28068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:41541 every 8 connection(s)
I20260809 12:17:04.742871 27982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:17:04.746150 28069 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:04.749073 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 27982
I20260809 12:17:04.749274 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:17:04.750073 20191 external_mini_cluster.cc:1428] Setting key 16fb0e36f13a3c140a5eaa448f345451
I20260809 12:17:04.758083 28069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c16d917e3d4d2b962ee476f393365c: Bootstrap starting.
I20260809 12:17:04.761842 28069 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 60c16d917e3d4d2b962ee476f393365c: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:04.763288 28069 log.cc:824] T 00000000000000000000000000000000 P 60c16d917e3d4d2b962ee476f393365c: Log is configured to *not* fsync() on all Append() calls
I20260809 12:17:04.764745 28072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:04.751871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55562 (local address 127.19.183.254:41541)
0809 12:17:04.752809 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:04.752819 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:04.752857 (+ 38us) server_negotiation.cc:407] Connection header received
0809 12:17:04.754485 (+ 1628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:04.754505 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:04.754996 (+ 491us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:04.755297 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:04.755879 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:04.757070 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:04.758832 (+ 1762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:04.759528 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:04.759658 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:04.761397 (+ 1739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:04.761409 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:04.761425 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:04.762156 (+ 731us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:04.762302 (+ 146us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:04.762347 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:04.762473 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:04.762839 (+ 366us) server_negotiation.cc:299] Negotiation successful
0809 12:17:04.763347 (+ 508us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":395,"threads_started":1}
I20260809 12:17:04.766137 28069 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 60c16d917e3d4d2b962ee476f393365c: No bootstrap required, opened a new log
I20260809 12:17:04.768282 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:17:04.773751 28069 raft_consensus.cc:348] T 00000000000000000000000000000000 P 60c16d917e3d4d2b962ee476f393365c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c16d917e3d4d2b962ee476f393365c" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41541 } }
I20260809 12:17:04.774209 28069 raft_consensus.cc:374] T 00000000000000000000000000000000 P 60c16d917e3d4d2b962ee476f393365c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:04.774361 28069 raft_consensus.cc:729] T 00000000000000000000000000000000 P 60c16d917e3d4d2b962ee476f393365c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 60c16d917e3d4d2b962ee476f393365c, State: Initialized, Role: FOLLOWER
I20260809 12:17:04.774971 28069 consensus_queue.cc:260] T 00000000000000000000000000000000 P 60c16d917e3d4d2b962ee476f393365c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c16d917e3d4d2b962ee476f393365c" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41541 } }
I20260809 12:17:04.775218 28069 raft_consensus.cc:388] T 00000000000000000000000000000000 P 60c16d917e3d4d2b962ee476f393365c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:17:04.775355 28069 raft_consensus.cc:482] T 00000000000000000000000000000000 P 60c16d917e3d4d2b962ee476f393365c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:17:04.775519 28069 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 60c16d917e3d4d2b962ee476f393365c [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:04.777485 28069 raft_consensus.cc:504] T 00000000000000000000000000000000 P 60c16d917e3d4d2b962ee476f393365c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c16d917e3d4d2b962ee476f393365c" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41541 } }
I20260809 12:17:04.777957 28069 leader_election.cc:302] T 00000000000000000000000000000000 P 60c16d917e3d4d2b962ee476f393365c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 60c16d917e3d4d2b962ee476f393365c; no voters:
I20260809 12:17:04.778621 28069 leader_election.cc:288] T 00000000000000000000000000000000 P 60c16d917e3d4d2b962ee476f393365c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:17:04.778795 28075 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 60c16d917e3d4d2b962ee476f393365c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:17:04.779515 28075 raft_consensus.cc:686] T 00000000000000000000000000000000 P 60c16d917e3d4d2b962ee476f393365c [term 1 LEADER]: Becoming Leader. State: Replica: 60c16d917e3d4d2b962ee476f393365c, State: Running, Role: LEADER
I20260809 12:17:04.780054 28075 consensus_queue.cc:237] T 00000000000000000000000000000000 P 60c16d917e3d4d2b962ee476f393365c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c16d917e3d4d2b962ee476f393365c" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41541 } }
I20260809 12:17:04.780615 28069 sys_catalog.cc:564] T 00000000000000000000000000000000 P 60c16d917e3d4d2b962ee476f393365c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:17:04.782687 28076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c16d917e3d4d2b962ee476f393365c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "60c16d917e3d4d2b962ee476f393365c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c16d917e3d4d2b962ee476f393365c" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41541 } } }
I20260809 12:17:04.783048 28076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c16d917e3d4d2b962ee476f393365c [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:04.783601 28077 sys_catalog.cc:455] T 00000000000000000000000000000000 P 60c16d917e3d4d2b962ee476f393365c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 60c16d917e3d4d2b962ee476f393365c. Latest consensus state: current_term: 1 leader_uuid: "60c16d917e3d4d2b962ee476f393365c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "60c16d917e3d4d2b962ee476f393365c" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 41541 } } }
I20260809 12:17:04.783944 28077 sys_catalog.cc:458] T 00000000000000000000000000000000 P 60c16d917e3d4d2b962ee476f393365c [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:04.784047 28082 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:17:04.790302 28082 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:17:04.798002 28082 catalog_manager.cc:1347] Generated new cluster ID: c3808f17453c4f77ab7e7efe0432593a
I20260809 12:17:04.798125 28082 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:17:04.816529 28082 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:17:04.817610 28082 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:17:04.828248 28082 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 60c16d917e3d4d2b962ee476f393365c: Generated new TSK 0
I20260809 12:17:04.828962 28082 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:17:05.780381 28072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:04.846156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55576 (local address 127.19.183.254:41541)
0809 12:17:04.846338 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:04.846345 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:04.846803 (+ 458us) server_negotiation.cc:407] Connection header received
0809 12:17:04.847031 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:04.847036 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:04.847132 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:04.847288 (+ 156us) server_negotiation.cc:226] Negotiated authn=JWT
0809 12:17:04.847703 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:04.848960 (+ 1257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:04.850338 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:04.851028 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:04.851149 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:04.851580 (+ 431us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 12:17:05.779815 (+928235us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 12:17:05.780151 (+ 336us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55576: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 12:17:05.780481 28072 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55576: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 12:17:05.780762 28108 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:04.846017 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:41541 (local address 127.0.0.1:55576)
0809 12:17:04.846666 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:04.846681 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:04.846854 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:04.847296 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:04.847300 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:04.847308 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0809 12:17:04.847538 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:04.847546 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:04.849113 (+ 1567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:04.849117 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:04.850200 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:04.850211 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:04.851199 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:04.851203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:04.851317 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:04.851348 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 12:17:05.780109 (+928761us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 12:17:05.780367 (+ 258us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.19.183.254:41541: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":523,"thread_start_us":368,"threads_started":1}
I20260809 12:17:05.782048 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 27982
I20260809 12:17:05.783267 28063 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:05.884353 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 27982
2026-08-09T12:17:05Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (1636 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (1549 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (426 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:17:07 dist-test-slave-39l8 krb5kdc[28126](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:17:07 dist-test-slave-39l8 krb5kdc[28126](info): set up 2 sockets
Aug 09 12:17:07 dist-test-slave-39l8 krb5kdc[28126](info): commencing operation
krb5kdc: starting...
W20260809 12:17:09.138134 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.229s user 0.003s sys 0.027s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:17:09 dist-test-slave-39l8 krb5kdc[28126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:17:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:17:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:17:09.382099 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:44737
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:34593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:44737
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:17:09.717522 28150 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:09.717820 28150 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:09.717911 28150 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:09.725082 28150 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:17:09.725189 28150 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:09.725234 28150 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:09.725292 28150 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:17:09.725329 28150 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:17:09.735220 28150 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:34593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:44737
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:44737
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:09.736446 28150 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:09.737668 28150 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:09.749372 28165 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:09.749473 28160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:09.749401 28163 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:09.750396 28150 server_base.cc:1034] running on GCE node
I20260809 12:17:09.751261 28150 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:09.752714 28150 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:09.753961 28150 hybrid_clock.cc:648] HybridClock initialized: now 1786277829753913 us; error 55 us; skew 500 ppm
I20260809 12:17:09.754318 28150 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:09 dist-test-slave-39l8 krb5kdc[28126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277829, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:09.761826 28150 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:17:09.763311 28150 webserver.cc:460] Webserver started at http://127.19.183.254:34229/ using document root <none> and password file <none>
I20260809 12:17:09.763994 28150 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:09.764119 28150 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:09.764384 28150 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:09.767562 28150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "c3f504d2bd564febb4da3b5c6b50037f"
format_stamp: "Formatted at 2026-08-09 12:17:09 on dist-test-slave-39l8"
server_key: "4f7ae81b141c3e2470287955b37fe1b2"
server_key_iv: "b44adb7876d2eb9db0f26b42f5b189c9"
server_key_version: "encryptionkey@0"
I20260809 12:17:09.768329 28150 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "c3f504d2bd564febb4da3b5c6b50037f"
format_stamp: "Formatted at 2026-08-09 12:17:09 on dist-test-slave-39l8"
server_key: "4f7ae81b141c3e2470287955b37fe1b2"
server_key_iv: "b44adb7876d2eb9db0f26b42f5b189c9"
server_key_version: "encryptionkey@0"
I20260809 12:17:09.773802 28150 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:17:09.777304 28175 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:09.778676 28150 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:17:09.778949 28150 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "c3f504d2bd564febb4da3b5c6b50037f"
format_stamp: "Formatted at 2026-08-09 12:17:09 on dist-test-slave-39l8"
server_key: "4f7ae81b141c3e2470287955b37fe1b2"
server_key_iv: "b44adb7876d2eb9db0f26b42f5b189c9"
server_key_version: "encryptionkey@0"
I20260809 12:17:09.779181 28150 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:09.794659 28150 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:09.798635 28150 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:09.798983 28150 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:09.818035 28150 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:44737
I20260809 12:17:09.818053 28240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:44737 every 8 connection(s)
I20260809 12:17:09.819543 28150 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:17:09.823323 28242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:09.825966 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 28150
I20260809 12:17:09.826226 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:17:09.826938 20191 external_mini_cluster.cc:1428] Setting key 6550c2313e36140e5a02537f9955cb98
I20260809 12:17:09.835793 28242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3f504d2bd564febb4da3b5c6b50037f: Bootstrap starting.
I20260809 12:17:09.839793 28242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c3f504d2bd564febb4da3b5c6b50037f: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:09.841255 28242 log.cc:824] T 00000000000000000000000000000000 P c3f504d2bd564febb4da3b5c6b50037f: Log is configured to *not* fsync() on all Append() calls
Aug 09 12:17:09 dist-test-slave-39l8 krb5kdc[28126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277829, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:17:09.844372 28242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c3f504d2bd564febb4da3b5c6b50037f: No bootstrap required, opened a new log
I20260809 12:17:09.851508 28247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:09.828919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44112 (local address 127.19.183.254:44737)
0809 12:17:09.829723 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:09.829734 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:09.829775 (+ 41us) server_negotiation.cc:407] Connection header received
0809 12:17:09.831553 (+ 1778us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:09.831577 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:09.832003 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:09.832362 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:09.834207 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:09.835630 (+ 1423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:09.837411 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:09.838117 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:09.838252 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:09.843515 (+ 5263us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:09.843524 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:09.843540 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:09.843580 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:09.847056 (+ 3476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:09.847808 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:09.847815 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:09.847821 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:09.848011 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:09.848411 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:09.848426 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:09.848428 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:09.849000 (+ 572us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:09.849332 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:09.849856 (+ 524us) server_negotiation.cc:299] Negotiation successful
0809 12:17:09.850331 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":590,"thread_start_us":368,"threads_started":1}
I20260809 12:17:09.852707 28242 raft_consensus.cc:348] T 00000000000000000000000000000000 P c3f504d2bd564febb4da3b5c6b50037f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f504d2bd564febb4da3b5c6b50037f" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44737 } }
I20260809 12:17:09.853204 28242 raft_consensus.cc:374] T 00000000000000000000000000000000 P c3f504d2bd564febb4da3b5c6b50037f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:09.853327 28242 raft_consensus.cc:729] T 00000000000000000000000000000000 P c3f504d2bd564febb4da3b5c6b50037f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c3f504d2bd564febb4da3b5c6b50037f, State: Initialized, Role: FOLLOWER
I20260809 12:17:09.853906 28242 consensus_queue.cc:260] T 00000000000000000000000000000000 P c3f504d2bd564febb4da3b5c6b50037f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f504d2bd564febb4da3b5c6b50037f" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44737 } }
I20260809 12:17:09.854192 28242 raft_consensus.cc:388] T 00000000000000000000000000000000 P c3f504d2bd564febb4da3b5c6b50037f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:17:09.854352 28242 raft_consensus.cc:482] T 00000000000000000000000000000000 P c3f504d2bd564febb4da3b5c6b50037f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:17:09.854575 28242 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c3f504d2bd564febb4da3b5c6b50037f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:09.856848 28242 raft_consensus.cc:504] T 00000000000000000000000000000000 P c3f504d2bd564febb4da3b5c6b50037f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f504d2bd564febb4da3b5c6b50037f" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44737 } }
I20260809 12:17:09.857340 28242 leader_election.cc:302] T 00000000000000000000000000000000 P c3f504d2bd564febb4da3b5c6b50037f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c3f504d2bd564febb4da3b5c6b50037f; no voters:
I20260809 12:17:09.858039 28242 leader_election.cc:288] T 00000000000000000000000000000000 P c3f504d2bd564febb4da3b5c6b50037f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:17:09.858150 28249 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c3f504d2bd564febb4da3b5c6b50037f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:17:09.858986 28249 raft_consensus.cc:686] T 00000000000000000000000000000000 P c3f504d2bd564febb4da3b5c6b50037f [term 1 LEADER]: Becoming Leader. State: Replica: c3f504d2bd564febb4da3b5c6b50037f, State: Running, Role: LEADER
I20260809 12:17:09.859548 28249 consensus_queue.cc:237] T 00000000000000000000000000000000 P c3f504d2bd564febb4da3b5c6b50037f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f504d2bd564febb4da3b5c6b50037f" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44737 } }
I20260809 12:17:09.860205 28242 sys_catalog.cc:564] T 00000000000000000000000000000000 P c3f504d2bd564febb4da3b5c6b50037f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:17:09.862494 28252 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3f504d2bd564febb4da3b5c6b50037f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c3f504d2bd564febb4da3b5c6b50037f. Latest consensus state: current_term: 1 leader_uuid: "c3f504d2bd564febb4da3b5c6b50037f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f504d2bd564febb4da3b5c6b50037f" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44737 } } }
I20260809 12:17:09.862882 28252 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3f504d2bd564febb4da3b5c6b50037f [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:09.864322 28251 sys_catalog.cc:455] T 00000000000000000000000000000000 P c3f504d2bd564febb4da3b5c6b50037f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c3f504d2bd564febb4da3b5c6b50037f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c3f504d2bd564febb4da3b5c6b50037f" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44737 } } }
I20260809 12:17:09.864681 28251 sys_catalog.cc:458] T 00000000000000000000000000000000 P c3f504d2bd564febb4da3b5c6b50037f [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:09.865155 28260 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:17:09.871747 28260 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:17:09.880982 28260 catalog_manager.cc:1347] Generated new cluster ID: 7ab339a2d0a4451aa6669a9922cc8d0c
I20260809 12:17:09.881091 28260 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:17:09.949661 28260 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:17:09.951583 28260 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:17:09.959889 28260 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c3f504d2bd564febb4da3b5c6b50037f: Generated new TSK 0
I20260809 12:17:09.960896 28260 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
I20260809 12:17:10.032366 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:44737
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:34593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:17:10.375888 28283 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:10.376159 28283 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:10.376259 28283 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:10.383239 28283 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:10.383342 28283 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:10.383451 28283 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:17:10.393016 28283 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:34593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:44737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:10.394328 28283 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:10.395601 28283 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:10.408838 28288 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:10.408883 28289 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:10.408924 28291 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:10.409883 28283 server_base.cc:1034] running on GCE node
I20260809 12:17:10.410471 28283 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:10.411514 28283 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:10.412724 28283 hybrid_clock.cc:648] HybridClock initialized: now 1786277830412664 us; error 50 us; skew 500 ppm
I20260809 12:17:10.412986 28283 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:10 dist-test-slave-39l8 krb5kdc[28126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277830, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:10.416186 28283 init.cc:377] Logged in from keytab as kudu/127.19.183.193@KRBTEST.COM (short username kudu)
I20260809 12:17:10.417603 28283 webserver.cc:460] Webserver started at http://127.19.183.193:37945/ using document root <none> and password file <none>
I20260809 12:17:10.418218 28283 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:10.418318 28283 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:10.418594 28283 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:10.421527 28283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "eda690372f9e4e4f885f3c7a52de813f"
format_stamp: "Formatted at 2026-08-09 12:17:10 on dist-test-slave-39l8"
server_key: "e22c4a3c1094f5b08b0922d564bc8a0f"
server_key_iv: "1656f266a3138667d4a0551e9eb8fc22"
server_key_version: "encryptionkey@0"
I20260809 12:17:10.422245 28283 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "eda690372f9e4e4f885f3c7a52de813f"
format_stamp: "Formatted at 2026-08-09 12:17:10 on dist-test-slave-39l8"
server_key: "e22c4a3c1094f5b08b0922d564bc8a0f"
server_key_iv: "1656f266a3138667d4a0551e9eb8fc22"
server_key_version: "encryptionkey@0"
I20260809 12:17:10.427130 28283 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 12:17:10.430495 28303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:10.431833 28283 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:17:10.431967 28283 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "eda690372f9e4e4f885f3c7a52de813f"
format_stamp: "Formatted at 2026-08-09 12:17:10 on dist-test-slave-39l8"
server_key: "e22c4a3c1094f5b08b0922d564bc8a0f"
server_key_iv: "1656f266a3138667d4a0551e9eb8fc22"
server_key_version: "encryptionkey@0"
I20260809 12:17:10.432178 28283 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:10.447060 28283 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:10.451195 28283 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:10.451429 28283 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:10.452549 28283 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:10.453900 28283 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:10.453976 28283 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:10.454082 28283 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:10.454135 28283 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:10.487375 28283 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:43673
I20260809 12:17:10.487424 28458 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:43673 every 8 connection(s)
I20260809 12:17:10.488760 28283 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:17:10.498870 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 28283
I20260809 12:17:10.499097 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:17:10.499927 20191 external_mini_cluster.cc:1428] Setting key c80660163abedf9aa12308ff4e96a025
Aug 09 12:17:10 dist-test-slave-39l8 krb5kdc[28126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277830, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:17:10.511096 28461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:10.491467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:57763 (local address 127.19.183.254:44737)
0809 12:17:10.495727 (+ 4260us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:10.495732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:10.495747 (+ 15us) server_negotiation.cc:407] Connection header received
0809 12:17:10.495821 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:10.495824 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:10.495883 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:10.495969 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:10.497446 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:10.498137 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:10.500594 (+ 2457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:10.501153 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:10.501275 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:10.504766 (+ 3491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:10.504771 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:10.504774 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:10.504807 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:10.507682 (+ 2875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:10.508557 (+ 875us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:10.508564 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:10.508570 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:10.508698 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:10.509166 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:10.509173 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:10.509178 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:10.509515 (+ 337us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:10.509643 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:10.510606 (+ 963us) server_negotiation.cc:299] Negotiation successful
0809 12:17:10.510837 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4132,"thread_start_us":4008,"threads_started":1}
I20260809 12:17:10.512409 28463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:10.491882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:44737 (local address 127.19.183.193:57763)
0809 12:17:10.493259 (+ 1377us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:10.493304 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:10.495184 (+ 1880us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:10.496206 (+ 1022us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:10.496214 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:10.496729 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:10.497276 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:10.497293 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:10.498292 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:10.498296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:10.500397 (+ 2101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:10.500412 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:10.501352 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:10.501358 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:10.501573 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:10.502300 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:10.502323 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:10.504553 (+ 2230us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:10.507839 (+ 3286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:10.507850 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:10.507866 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:10.508411 (+ 545us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:10.508812 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:10.508818 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:10.508824 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:10.509042 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:10.509669 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:10.509680 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:10.510144 (+ 464us) client_negotiation.cc:769] Sending connection context
0809 12:17:10.510475 (+ 331us) client_negotiation.cc:240] Negotiation successful
0809 12:17:10.511071 (+ 596us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1094,"thread_start_us":885,"threads_started":1}
I20260809 12:17:10.514096 28459 heartbeater.cc:347] Connected to a master server at 127.19.183.254:44737
I20260809 12:17:10.514408 28459 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:10.515223 28459 heartbeater.cc:511] Master 127.19.183.254:44737 requested a full tablet report, sending...
I20260809 12:17:10.517287 28196 ts_manager.cc:194] Registered new tserver with Master: eda690372f9e4e4f885f3c7a52de813f (127.19.183.193:43673)
I20260809 12:17:10.518673 28196 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:57763
WARNING: no policy specified for kudu/127.19.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
I20260809 12:17:10.555122 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:44737
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:34593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:17:10.890321 28467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:10.890647 28467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:10.890852 28467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:10.897890 28467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:10.898052 28467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:10.898241 28467 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:17:10.907604 28467 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:34593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:44737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:10.908924 28467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:10.910336 28467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:10.924065 28473 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:10.923970 28475 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:10.924422 28477 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:10.925026 28467 server_base.cc:1034] running on GCE node
I20260809 12:17:10.925621 28467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:10.926548 28467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:10.927806 28467 hybrid_clock.cc:648] HybridClock initialized: now 1786277830927755 us; error 60 us; skew 500 ppm
I20260809 12:17:10.928171 28467 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:10 dist-test-slave-39l8 krb5kdc[28126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277830, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:10.931474 28467 init.cc:377] Logged in from keytab as kudu/127.19.183.194@KRBTEST.COM (short username kudu)
I20260809 12:17:10.932837 28467 webserver.cc:460] Webserver started at http://127.19.183.194:35393/ using document root <none> and password file <none>
I20260809 12:17:10.933499 28467 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:10.933600 28467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:10.933883 28467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:10.937096 28467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "13a553730f8c498abc8263eb272ceb49"
format_stamp: "Formatted at 2026-08-09 12:17:10 on dist-test-slave-39l8"
server_key: "66032d619a79abb19ce5ed33f5edea19"
server_key_iv: "f9b8b4bddd0f02fbd9e09dc1e8e75eab"
server_key_version: "encryptionkey@0"
I20260809 12:17:10.937911 28467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "13a553730f8c498abc8263eb272ceb49"
format_stamp: "Formatted at 2026-08-09 12:17:10 on dist-test-slave-39l8"
server_key: "66032d619a79abb19ce5ed33f5edea19"
server_key_iv: "f9b8b4bddd0f02fbd9e09dc1e8e75eab"
server_key_version: "encryptionkey@0"
I20260809 12:17:10.943169 28467 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.005s
I20260809 12:17:10.946666 28486 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:10.948036 28467 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:17:10.948220 28467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "13a553730f8c498abc8263eb272ceb49"
format_stamp: "Formatted at 2026-08-09 12:17:10 on dist-test-slave-39l8"
server_key: "66032d619a79abb19ce5ed33f5edea19"
server_key_iv: "f9b8b4bddd0f02fbd9e09dc1e8e75eab"
server_key_version: "encryptionkey@0"
I20260809 12:17:10.948442 28467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:10.971154 28467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:10.975209 28467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:10.975507 28467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:10.976512 28467 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:10.977867 28467 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:10.977981 28467 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:10.978145 28467 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:10.978222 28467 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:11.012604 28467 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:34823
I20260809 12:17:11.012701 28647 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:34823 every 8 connection(s)
I20260809 12:17:11.014045 28467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:17:11.018795 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 28467
I20260809 12:17:11.018960 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:17:11.019423 20191 external_mini_cluster.cc:1428] Setting key 4c29074bb053819bb6cfc719dfc7c033
Aug 09 12:17:11 dist-test-slave-39l8 krb5kdc[28126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277830, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:17:11.034174 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:11.016663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:60771 (local address 127.19.183.254:44737)
0809 12:17:11.017106 (+ 443us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:11.017111 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:11.018062 (+ 951us) server_negotiation.cc:407] Connection header received
0809 12:17:11.020188 (+ 2126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:11.020193 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:11.020288 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:11.020418 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:11.021915 (+ 1497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:11.022638 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:11.024232 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:11.024668 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:11.024801 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:11.028315 (+ 3514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:11.028320 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:11.028323 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:11.028355 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:11.031054 (+ 2699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:11.031802 (+ 748us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:11.031808 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:11.031812 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:11.031949 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:11.032408 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:11.032413 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:11.032417 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:11.032690 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:11.032863 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:11.033734 (+ 871us) server_negotiation.cc:299] Negotiation successful
0809 12:17:11.033906 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":220,"threads_started":1}
I20260809 12:17:11.035861 28653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:11.017180 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:44737 (local address 127.19.183.194:60771)
0809 12:17:11.017891 (+ 711us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:11.017938 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:11.019908 (+ 1970us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:11.020672 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:11.020680 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:11.021176 (+ 496us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:11.021736 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:11.021754 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:11.022830 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:11.022834 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:11.024050 (+ 1216us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:11.024064 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:11.024910 (+ 846us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:11.024917 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:11.025150 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:11.025840 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:11.025863 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:11.028091 (+ 2228us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:11.031203 (+ 3112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:11.031212 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:11.031227 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:11.031669 (+ 442us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:11.032082 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:11.032087 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:11.032091 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:11.032279 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:11.032826 (+ 547us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:11.032834 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:11.033299 (+ 465us) client_negotiation.cc:769] Sending connection context
0809 12:17:11.033645 (+ 346us) client_negotiation.cc:240] Negotiation successful
0809 12:17:11.034230 (+ 585us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"thread_start_us":281,"threads_started":1}
I20260809 12:17:11.037432 28648 heartbeater.cc:347] Connected to a master server at 127.19.183.254:44737
I20260809 12:17:11.037750 28648 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:11.038453 28648 heartbeater.cc:511] Master 127.19.183.254:44737 requested a full tablet report, sending...
I20260809 12:17:11.040040 28196 ts_manager.cc:194] Registered new tserver with Master: 13a553730f8c498abc8263eb272ceb49 (127.19.183.194:34823)
I20260809 12:17:11.040884 28196 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:60771
WARNING: no policy specified for kudu/127.19.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
I20260809 12:17:11.079434 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:44737
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:34593
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:17:11.414332 28656 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:11.414631 28656 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:11.414826 28656 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:11.422330 28656 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:11.422458 28656 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:11.422624 28656 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:17:11.431805 28656 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:34593
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:44737
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:11.433094 28656 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:11.434475 28656 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:11.447094 28664 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:11.447338 28667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:11.447693 28663 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:11.448091 28656 server_base.cc:1034] running on GCE node
I20260809 12:17:11.448619 28656 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:11.449420 28656 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:11.450608 28656 hybrid_clock.cc:648] HybridClock initialized: now 1786277831450551 us; error 54 us; skew 500 ppm
I20260809 12:17:11.450947 28656 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:11 dist-test-slave-39l8 krb5kdc[28126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277831, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:11.453984 28656 init.cc:377] Logged in from keytab as kudu/127.19.183.195@KRBTEST.COM (short username kudu)
I20260809 12:17:11.455417 28656 webserver.cc:460] Webserver started at http://127.19.183.195:41659/ using document root <none> and password file <none>
I20260809 12:17:11.456081 28656 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:11.456230 28656 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:11.456532 28656 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:11.459425 28656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "a86d458bc21442d6b6da1e34387bb808"
format_stamp: "Formatted at 2026-08-09 12:17:11 on dist-test-slave-39l8"
server_key: "43a44df5981260e4ab1ace5296ff9dc3"
server_key_iv: "98ec0ef39616ee8faa15357acd831e66"
server_key_version: "encryptionkey@0"
I20260809 12:17:11.460317 28656 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "a86d458bc21442d6b6da1e34387bb808"
format_stamp: "Formatted at 2026-08-09 12:17:11 on dist-test-slave-39l8"
server_key: "43a44df5981260e4ab1ace5296ff9dc3"
server_key_iv: "98ec0ef39616ee8faa15357acd831e66"
server_key_version: "encryptionkey@0"
I20260809 12:17:11.465175 28656 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260809 12:17:11.468549 28676 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:11.469760 28656 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:17:11.469923 28656 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "a86d458bc21442d6b6da1e34387bb808"
format_stamp: "Formatted at 2026-08-09 12:17:11 on dist-test-slave-39l8"
server_key: "43a44df5981260e4ab1ace5296ff9dc3"
server_key_iv: "98ec0ef39616ee8faa15357acd831e66"
server_key_version: "encryptionkey@0"
I20260809 12:17:11.470124 28656 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:11.488785 28656 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:11.492691 28656 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:11.492977 28656 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:11.493865 28656 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:11.495172 28656 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:11.495288 28656 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:11.495426 28656 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:11.495497 28656 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:11.521596 28459 heartbeater.cc:503] Master 127.19.183.254:44737 was elected leader, sending a full tablet report...
I20260809 12:17:11.529090 28656 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:37379
I20260809 12:17:11.529176 28824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:37379 every 8 connection(s)
I20260809 12:17:11.530411 28656 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:17:11.535199 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 28656
I20260809 12:17:11.535372 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:17:11.535863 20191 external_mini_cluster.cc:1428] Setting key 698e67dfb2384ace8130e478bcd5b7e9
Aug 09 12:17:11 dist-test-slave-39l8 krb5kdc[28126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277831, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:17:11.549826 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:11.533129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:49389 (local address 127.19.183.254:44737)
0809 12:17:11.533334 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:11.533338 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:11.534517 (+ 1179us) server_negotiation.cc:407] Connection header received
0809 12:17:11.536437 (+ 1920us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:11.536444 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:11.536537 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:11.536636 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:11.538324 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:11.539242 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:11.540775 (+ 1533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:11.541178 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:11.541308 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:11.544583 (+ 3275us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:11.544588 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:11.544591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:11.544622 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:11.546953 (+ 2331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:11.547634 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:11.547640 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:11.547643 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:11.547776 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:11.548186 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:11.548191 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:11.548194 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:11.548437 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:11.548632 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:11.549220 (+ 588us) server_negotiation.cc:299] Negotiation successful
0809 12:17:11.549434 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 12:17:11.550805 28827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:11.533642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:44737 (local address 127.19.183.195:49389)
0809 12:17:11.534362 (+ 720us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:11.534398 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:11.536173 (+ 1775us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:11.536919 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:11.536928 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:11.537519 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:11.538151 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:11.538170 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:11.539397 (+ 1227us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:11.539401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:11.540597 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:11.540610 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:11.541402 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:11.541409 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:11.541626 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:11.542220 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:11.542245 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:11.544387 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:11.547089 (+ 2702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:11.547097 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:11.547111 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:11.547507 (+ 396us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:11.547887 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:11.547892 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:11.547895 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:11.548064 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:11.548587 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:11.548594 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:11.548894 (+ 300us) client_negotiation.cc:769] Sending connection context
0809 12:17:11.549142 (+ 248us) client_negotiation.cc:240] Negotiation successful
0809 12:17:11.549645 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":486,"thread_start_us":286,"threads_started":1}
I20260809 12:17:11.552217 28825 heartbeater.cc:347] Connected to a master server at 127.19.183.254:44737
I20260809 12:17:11.552479 28825 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:11.553129 28825 heartbeater.cc:511] Master 127.19.183.254:44737 requested a full tablet report, sending...
I20260809 12:17:11.554618 28196 ts_manager.cc:194] Registered new tserver with Master: a86d458bc21442d6b6da1e34387bb808 (127.19.183.195:37379)
I20260809 12:17:11.555406 28196 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:49389
I20260809 12:17:11.563120 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/test-admin.keytab.
Aug 09 12:17:11 dist-test-slave-39l8 krb5kdc[28126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:11.595945 20191 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 12:17:11.601482 28835 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:11.598659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34202 (local address 127.19.183.193:43673)
0809 12:17:11.599336 (+ 677us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:11.599343 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:11.599370 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:17:11.599435 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:11.599443 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:11.599672 (+ 229us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:11.599837 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:11.601033 (+ 1196us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34202: BlockingRecv error: recv got EOF from 127.0.0.1:34202 (error 108)
Metrics: {"server-negotiator.queue_time_us":511,"thread_start_us":349,"threads_started":1}
W20260809 12:17:11.601475 20191 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/krb5cc' of type FILE
Aug 09 12:17:11 dist-test-slave-39l8 krb5kdc[28126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:11.603119 20191 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 12:17:11 dist-test-slave-39l8 krb5kdc[28126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:11.604434 20191 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 12:17:11 dist-test-slave-39l8 krb5kdc[28126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.193@KRBTEST.COM
I20260809 12:17:11.619627 28835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:11.605925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34216 (local address 127.19.183.193:43673)
0809 12:17:11.606166 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:11.606173 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:11.606405 (+ 232us) server_negotiation.cc:407] Connection header received
0809 12:17:11.606614 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:11.606623 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:11.606679 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:11.606802 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:11.607764 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:11.608690 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:11.609933 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:11.610489 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:11.610642 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:11.614144 (+ 3502us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:11.614154 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:11.614168 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:11.614198 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:11.616852 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:11.617568 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:11.617575 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:11.617577 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:11.617697 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:11.618075 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:11.618079 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:11.618081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:11.618414 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:11.618745 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:11.619093 (+ 348us) server_negotiation.cc:299] Negotiation successful
0809 12:17:11.619363 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260809 12:17:11.624953 28835 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:11.623457 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34232 (local address 127.19.183.193:43673)
0809 12:17:11.623659 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:11.623665 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:11.623762 (+ 97us) server_negotiation.cc:407] Connection header received
0809 12:17:11.623890 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:11.623893 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:11.623956 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:11.624028 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:11.624742 (+ 714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34232: BlockingRecv error: recv got EOF from 127.0.0.1:34232 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 12:17:11.625085 20191 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/krb5cc' of type FILE
Aug 09 12:17:11 dist-test-slave-39l8 krb5kdc[28126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:11.626576 20191 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 12:17:11 dist-test-slave-39l8 krb5kdc[28126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:11.627846 20191 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 12:17:11 dist-test-slave-39l8 krb5kdc[28126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.193@KRBTEST.COM
I20260809 12:17:11.641227 28835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:11.629063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34242 (local address 127.19.183.193:43673)
0809 12:17:11.629237 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:11.629243 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:11.629443 (+ 200us) server_negotiation.cc:407] Connection header received
0809 12:17:11.629628 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:11.629633 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:11.629684 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:11.629780 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:11.630683 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:11.631423 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:11.632621 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:11.633038 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:11.633183 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:11.636552 (+ 3369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:11.636556 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:11.636559 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:11.636600 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:11.638844 (+ 2244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:11.639423 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:11.639429 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:11.639432 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:11.639570 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:11.639938 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:11.639943 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:11.639947 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:11.640227 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:11.640501 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:11.640770 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 12:17:11.640984 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
W20260809 12:17:11.644917 28835 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:11.643472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34258 (local address 127.19.183.193:43673)
0809 12:17:11.643694 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:11.643710 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:11.643876 (+ 166us) server_negotiation.cc:407] Connection header received
0809 12:17:11.643974 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:11.643977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:11.644035 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:11.644105 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:11.644730 (+ 625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34258: BlockingRecv error: recv got EOF from 127.0.0.1:34258 (error 108)
Metrics: {"server-negotiator.queue_time_us":108}
W20260809 12:17:11.645066 20191 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786277774779245-20191-0/krb5kdc/krb5cc' of type FILE
Aug 09 12:17:11 dist-test-slave-39l8 krb5kdc[28126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:11.646477 20191 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 12:17:11 dist-test-slave-39l8 krb5kdc[28126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:11.647733 20191 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 12:17:11 dist-test-slave-39l8 krb5kdc[28126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.193@KRBTEST.COM
I20260809 12:17:11.661296 28835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:11.649048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34270 (local address 127.19.183.193:43673)
0809 12:17:11.649237 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:11.649242 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:11.649341 (+ 99us) server_negotiation.cc:407] Connection header received
0809 12:17:11.649502 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:11.649511 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:11.649579 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:11.649677 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:11.650568 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:11.651300 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:11.652600 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:11.653011 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:11.653161 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:11.656447 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:11.656454 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:11.656458 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:11.656493 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:11.658745 (+ 2252us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:11.659332 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:11.659337 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:11.659341 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:11.659492 (+ 151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:11.659877 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:11.659881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:11.659885 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:11.660185 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:11.660497 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:11.660780 (+ 283us) server_negotiation.cc:299] Negotiation successful
0809 12:17:11.660987 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 12:17:11.662879 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 28283
I20260809 12:17:11.674166 28835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:11.663646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34276 (local address 127.19.183.193:43673)
0809 12:17:11.663827 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:11.663832 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:11.664044 (+ 212us) server_negotiation.cc:407] Connection header received
0809 12:17:11.664211 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:11.664216 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:11.664276 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:11.664369 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:11.665203 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:11.665793 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:11.667087 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:11.667500 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:11.667643 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:11.669439 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:11.669450 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:11.669455 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:11.669483 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:11.671676 (+ 2193us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:11.672294 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:11.672300 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:11.672304 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:11.672434 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:11.672796 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:11.672802 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:11.672805 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:11.673078 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:11.673354 (+ 276us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:11.673672 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 12:17:11.673866 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 12:17:11.674896 28452 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:11.757360 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 28283
I20260809 12:17:11.783377 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 28467
Aug 09 12:17:11 dist-test-slave-39l8 krb5kdc[28126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.194@KRBTEST.COM
I20260809 12:17:11.800225 28849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:11.784270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54102 (local address 127.19.183.194:34823)
0809 12:17:11.784921 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:11.784928 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:11.784953 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:17:11.785034 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:11.785041 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:11.785325 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:11.785492 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:11.786546 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:11.788053 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:11.789656 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:11.790536 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:11.790683 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:11.794391 (+ 3708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:11.794409 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:11.794423 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:11.794461 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:11.797306 (+ 2845us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:11.798021 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:11.798031 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:11.798034 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:11.798178 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:11.798549 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:11.798554 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:11.798557 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:11.798998 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:11.799327 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:11.799641 (+ 314us) server_negotiation.cc:299] Negotiation successful
0809 12:17:11.799907 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":339,"threads_started":1}
I20260809 12:17:11.801491 28639 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:11.889761 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 28467
I20260809 12:17:11.916968 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 28656
Aug 09 12:17:11 dist-test-slave-39l8 krb5kdc[28126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.195@KRBTEST.COM
I20260809 12:17:11.932832 28855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:11.917985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43724 (local address 127.19.183.195:37379)
0809 12:17:11.918635 (+ 650us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:11.918642 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:11.918670 (+ 28us) server_negotiation.cc:407] Connection header received
0809 12:17:11.918797 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:11.918804 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:11.919025 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:11.919202 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:11.920302 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:11.921288 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:11.922516 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:11.923182 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:11.923317 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:11.927289 (+ 3972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:11.927300 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:11.927314 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:11.927352 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:11.929971 (+ 2619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:11.930571 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:11.930578 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:11.930581 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:11.930700 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:11.931110 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:11.931114 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:11.931115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:11.931433 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:11.931780 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:11.932072 (+ 292us) server_negotiation.cc:299] Negotiation successful
0809 12:17:11.932365 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":478,"thread_start_us":327,"threads_started":1}
I20260809 12:17:11.934211 28818 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:12.013677 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 28656
I20260809 12:17:12.037281 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 28150
I20260809 12:17:12.038448 28233 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:12.140969 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 28150
2026-08-09T12:17:12Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (4284 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:17:12 dist-test-slave-39l8 krb5kdc[28862](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:17:12 dist-test-slave-39l8 krb5kdc[28862](info): set up 2 sockets
Aug 09 12:17:12 dist-test-slave-39l8 krb5kdc[28862](info): commencing operation
krb5kdc: starting...
W20260809 12:17:13.417097 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.223s user 0.001s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:17:13 dist-test-slave-39l8 krb5kdc[28862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:17:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:17:13Z Disabled control of system clock
WARNING: no policy specified for oryx/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.254.keytab.
Entry for principal oryx/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.254.keytab.
I20260809 12:17:13.653244 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:34413
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.254.keytab
--principal=oryx/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:42821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:34413
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:17:13.994786 28875 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:13.995049 28875 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:13.995244 28875 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:14.002245 28875 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:17:14.002357 28875 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:14.002444 28875 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:14.002543 28875 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:17:14.002621 28875 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:17:14.012616 28875 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:42821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:34413
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.254.keytab
--principal=oryx/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:34413
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:14.014410 28875 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:14.016211 28875 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:14.028188 28886 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:14.028271 28888 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:14.028247 28884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:14.029062 28875 server_base.cc:1034] running on GCE node
I20260809 12:17:14.029830 28875 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:14.031263 28875 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:14.032465 28875 hybrid_clock.cc:648] HybridClock initialized: now 1786277834032421 us; error 29 us; skew 500 ppm
I20260809 12:17:14.032797 28875 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:14 dist-test-slave-39l8 krb5kdc[28862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277834, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:14.035943 28875 init.cc:377] Logged in from keytab as oryx/127.19.183.254@KRBTEST.COM (short username oryx)
I20260809 12:17:14.037487 28875 webserver.cc:460] Webserver started at http://127.19.183.254:42799/ using document root <none> and password file <none>
I20260809 12:17:14.038174 28875 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:14.038321 28875 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:14.038626 28875 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:14.041572 28875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "7852578af3644424bdcd20194612fdac"
format_stamp: "Formatted at 2026-08-09 12:17:14 on dist-test-slave-39l8"
server_key: "8c62100822f4209186be577f025bdcd7"
server_key_iv: "50f7287035c69a07747ddcace73ae803"
server_key_version: "encryptionkey@0"
I20260809 12:17:14.042336 28875 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "7852578af3644424bdcd20194612fdac"
format_stamp: "Formatted at 2026-08-09 12:17:14 on dist-test-slave-39l8"
server_key: "8c62100822f4209186be577f025bdcd7"
server_key_iv: "50f7287035c69a07747ddcace73ae803"
server_key_version: "encryptionkey@0"
I20260809 12:17:14.047487 28875 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.004s
I20260809 12:17:14.050884 28895 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:14.052160 28875 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:17:14.052342 28875 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "7852578af3644424bdcd20194612fdac"
format_stamp: "Formatted at 2026-08-09 12:17:14 on dist-test-slave-39l8"
server_key: "8c62100822f4209186be577f025bdcd7"
server_key_iv: "50f7287035c69a07747ddcace73ae803"
server_key_version: "encryptionkey@0"
I20260809 12:17:14.052538 28875 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:14.065996 28875 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:14.069635 28875 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:14.069924 28875 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:14.088457 28875 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:34413
I20260809 12:17:14.088491 28964 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:34413 every 8 connection(s)
I20260809 12:17:14.089849 28875 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:17:14.093497 28965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:14.096136 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 28875
I20260809 12:17:14.096342 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:17:14.097054 20191 external_mini_cluster.cc:1428] Setting key a6483a2208de0abbac947d552871f6fd
I20260809 12:17:14.106485 28965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac: Bootstrap starting.
I20260809 12:17:14.109975 28965 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:14.111025 28965 log.cc:824] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac: Log is configured to *not* fsync() on all Append() calls
Aug 09 12:17:14 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM
I20260809 12:17:14.114228 28965 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac: No bootstrap required, opened a new log
I20260809 12:17:14.120831 28972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:14.099048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43222 (local address 127.19.183.254:34413)
0809 12:17:14.099852 (+ 804us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:14.099868 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:14.099917 (+ 49us) server_negotiation.cc:407] Connection header received
0809 12:17:14.102261 (+ 2344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:14.102290 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:14.102846 (+ 556us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:14.103289 (+ 443us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:14.104590 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:14.106477 (+ 1887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:14.107835 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:14.108909 (+ 1074us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:14.109101 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:14.113494 (+ 4393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:14.113501 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:14.113515 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:14.113554 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:14.116766 (+ 3212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:14.117586 (+ 820us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:14.117592 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:14.117598 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:14.117770 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:14.118179 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:14.118183 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:14.118185 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:14.118691 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:14.119005 (+ 314us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:14.119439 (+ 434us) server_negotiation.cc:299] Negotiation successful
0809 12:17:14.119815 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":559,"thread_start_us":330,"threads_started":1}
I20260809 12:17:14.123171 28965 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } }
I20260809 12:17:14.123569 28965 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:14.123688 28965 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7852578af3644424bdcd20194612fdac, State: Initialized, Role: FOLLOWER
I20260809 12:17:14.124212 28965 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } }
I20260809 12:17:14.124449 28965 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:17:14.124588 28965 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:17:14.124758 28965 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:14.126958 28965 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } }
I20260809 12:17:14.127465 28965 leader_election.cc:302] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7852578af3644424bdcd20194612fdac; no voters:
I20260809 12:17:14.128050 28965 leader_election.cc:288] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:17:14.128212 28976 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:17:14.129248 28976 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [term 1 LEADER]: Becoming Leader. State: Replica: 7852578af3644424bdcd20194612fdac, State: Running, Role: LEADER
I20260809 12:17:14.129834 28976 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } }
I20260809 12:17:14.130302 28965 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:17:14.132853 28981 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7852578af3644424bdcd20194612fdac. Latest consensus state: current_term: 1 leader_uuid: "7852578af3644424bdcd20194612fdac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } }
I20260809 12:17:14.133031 28977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7852578af3644424bdcd20194612fdac" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } }
I20260809 12:17:14.133301 28981 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:14.133328 28977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:14.134122 28988 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:17:14.139020 28988 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:17:14.149047 28988 catalog_manager.cc:1347] Generated new cluster ID: e474dd64bd7a4984a89c538000646418
I20260809 12:17:14.149187 28988 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:17:14.159541 28988 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:17:14.160756 28988 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:17:14.170068 28988 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac: Generated new TSK 0
I20260809 12:17:14.170913 28988 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.19.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.193.keytab.
Entry for principal oryx/127.19.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.193.keytab.
I20260809 12:17:14.241775 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:34413
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.193.keytab
--principal=oryx/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:42821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:17:14.591224 29004 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:14.591495 29004 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:14.591598 29004 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:14.598594 29004 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:14.598709 29004 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:14.598845 29004 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:17:14.608100 29004 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:42821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.193.keytab
--principal=oryx/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:34413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:14.609555 29004 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:14.610913 29004 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:14.623605 29014 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:14.623562 29015 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:14.623692 29017 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:14.624293 29004 server_base.cc:1034] running on GCE node
I20260809 12:17:14.624897 29004 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:14.625804 29004 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:14.627032 29004 hybrid_clock.cc:648] HybridClock initialized: now 1786277834626978 us; error 54 us; skew 500 ppm
I20260809 12:17:14.627337 29004 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:14 dist-test-slave-39l8 krb5kdc[28862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277834, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:14.630355 29004 init.cc:377] Logged in from keytab as oryx/127.19.183.193@KRBTEST.COM (short username oryx)
I20260809 12:17:14.631951 29004 webserver.cc:460] Webserver started at http://127.19.183.193:38489/ using document root <none> and password file <none>
I20260809 12:17:14.632649 29004 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:14.632795 29004 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:14.633105 29004 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:14.635953 29004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "10c712c9a0d94e829c9c5c5763aa3277"
format_stamp: "Formatted at 2026-08-09 12:17:14 on dist-test-slave-39l8"
server_key: "45f7528a1523223872b5d847a2c858b3"
server_key_iv: "c67dac44488dc2d500708d4301e4e91d"
server_key_version: "encryptionkey@0"
I20260809 12:17:14.636734 29004 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "10c712c9a0d94e829c9c5c5763aa3277"
format_stamp: "Formatted at 2026-08-09 12:17:14 on dist-test-slave-39l8"
server_key: "45f7528a1523223872b5d847a2c858b3"
server_key_iv: "c67dac44488dc2d500708d4301e4e91d"
server_key_version: "encryptionkey@0"
I20260809 12:17:14.641749 29004 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 12:17:14.645113 29026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:14.646412 29004 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 12:17:14.646586 29004 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "10c712c9a0d94e829c9c5c5763aa3277"
format_stamp: "Formatted at 2026-08-09 12:17:14 on dist-test-slave-39l8"
server_key: "45f7528a1523223872b5d847a2c858b3"
server_key_iv: "c67dac44488dc2d500708d4301e4e91d"
server_key_version: "encryptionkey@0"
I20260809 12:17:14.646872 29004 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:14.678048 29004 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:14.682528 29004 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:14.682904 29004 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:14.683866 29004 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:14.685246 29004 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:14.685349 29004 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:14.685500 29004 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:14.685571 29004 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:14.720084 29004 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:37655
I20260809 12:17:14.720137 29192 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:37655 every 8 connection(s)
I20260809 12:17:14.721386 29004 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:17:14.731295 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 29004
I20260809 12:17:14.731529 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:17:14.732285 20191 external_mini_cluster.cc:1428] Setting key 6fdd78a03f090812589ff26d88e27299
Aug 09 12:17:14 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277834, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM
I20260809 12:17:14.742364 29195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:14.723993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:53721 (local address 127.19.183.254:34413)
0809 12:17:14.724469 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:14.724474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:14.725274 (+ 800us) server_negotiation.cc:407] Connection header received
0809 12:17:14.727193 (+ 1919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:14.727199 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:14.727294 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:14.727411 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:14.728902 (+ 1491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:14.729647 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:14.731732 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:14.732203 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:14.732311 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:14.736602 (+ 4291us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:14.736606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:14.736610 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:14.736643 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:14.739175 (+ 2532us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:14.740099 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:14.740103 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:14.740105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:14.740227 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:14.740720 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:14.740723 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:14.740725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:14.740959 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:14.741067 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:14.741943 (+ 876us) server_negotiation.cc:299] Negotiation successful
0809 12:17:14.742105 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":376,"thread_start_us":255,"threads_started":1}
I20260809 12:17:14.743880 29196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:14.724438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.19.183.193:53721)
0809 12:17:14.725106 (+ 668us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:14.725156 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:14.726918 (+ 1762us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:14.727654 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:14.727661 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:14.728161 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:14.728720 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:14.728736 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:14.729819 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:14.729824 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:14.731555 (+ 1731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:14.731573 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:14.732462 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:14.732470 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:14.732766 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:14.733512 (+ 746us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:14.733545 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:14.736381 (+ 2836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:14.739312 (+ 2931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:14.739324 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:14.739340 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:14.739982 (+ 642us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:14.740332 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:14.740340 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:14.740346 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:14.740608 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:14.741084 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:14.741094 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:14.741559 (+ 465us) client_negotiation.cc:769] Sending connection context
0809 12:17:14.741863 (+ 304us) client_negotiation.cc:240] Negotiation successful
0809 12:17:14.742421 (+ 558us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":280,"threads_started":1}
I20260809 12:17:14.745457 29193 heartbeater.cc:347] Connected to a master server at 127.19.183.254:34413
I20260809 12:17:14.745805 29193 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:14.746853 29193 heartbeater.cc:511] Master 127.19.183.254:34413 requested a full tablet report, sending...
I20260809 12:17:14.748937 28920 ts_manager.cc:194] Registered new tserver with Master: 10c712c9a0d94e829c9c5c5763aa3277 (127.19.183.193:37655)
I20260809 12:17:14.750448 28920 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:53721
WARNING: no policy specified for oryx/127.19.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.194.keytab.
Entry for principal oryx/127.19.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.194.keytab.
I20260809 12:17:14.789984 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:34413
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.194.keytab
--principal=oryx/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:42821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:17:15.122530 29200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:15.122850 29200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:15.123015 29200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:15.129854 29200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:15.129964 29200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:15.130084 29200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:17:15.139446 29200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:42821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.194.keytab
--principal=oryx/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:34413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:15.140604 29200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:15.141870 29200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:15.154902 29208 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:15.154874 29209 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:15.155145 29211 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:15.155584 29200 server_base.cc:1034] running on GCE node
I20260809 12:17:15.156152 29200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:15.157078 29200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:15.158300 29200 hybrid_clock.cc:648] HybridClock initialized: now 1786277835158255 us; error 54 us; skew 500 ppm
I20260809 12:17:15.158598 29200 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:15 dist-test-slave-39l8 krb5kdc[28862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277835, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:15.165753 29200 init.cc:377] Logged in from keytab as oryx/127.19.183.194@KRBTEST.COM (short username oryx)
I20260809 12:17:15.167148 29200 webserver.cc:460] Webserver started at http://127.19.183.194:35481/ using document root <none> and password file <none>
I20260809 12:17:15.167760 29200 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:15.167863 29200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:15.168114 29200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:15.170889 29200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "2e535068581c4dbcaca1b42f72e5b911"
format_stamp: "Formatted at 2026-08-09 12:17:15 on dist-test-slave-39l8"
server_key: "692b3dcfe76f7ea41d1ffa7942b158bd"
server_key_iv: "2d2cc4dfd885d9f4a9ef9c6124b6a8b5"
server_key_version: "encryptionkey@0"
I20260809 12:17:15.171627 29200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "2e535068581c4dbcaca1b42f72e5b911"
format_stamp: "Formatted at 2026-08-09 12:17:15 on dist-test-slave-39l8"
server_key: "692b3dcfe76f7ea41d1ffa7942b158bd"
server_key_iv: "2d2cc4dfd885d9f4a9ef9c6124b6a8b5"
server_key_version: "encryptionkey@0"
I20260809 12:17:15.176544 29200 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 12:17:15.179787 29222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:15.180994 29200 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:17:15.181146 29200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "2e535068581c4dbcaca1b42f72e5b911"
format_stamp: "Formatted at 2026-08-09 12:17:15 on dist-test-slave-39l8"
server_key: "692b3dcfe76f7ea41d1ffa7942b158bd"
server_key_iv: "2d2cc4dfd885d9f4a9ef9c6124b6a8b5"
server_key_version: "encryptionkey@0"
I20260809 12:17:15.181303 29200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:15.210695 29200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:15.215132 29200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:15.215385 29200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:15.216295 29200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:15.217594 29200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:15.217666 29200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:15.217756 29200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:15.217808 29200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:15.251497 29200 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:42409
I20260809 12:17:15.251545 29383 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:42409 every 8 connection(s)
I20260809 12:17:15.252810 29200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:17:15.255486 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 29200
I20260809 12:17:15.255672 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:17:15.256191 20191 external_mini_cluster.cc:1428] Setting key 430117e5cd45548e3735d053689b7297
Aug 09 12:17:15 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277835, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM
I20260809 12:17:15.274663 29390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:15.255342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:59397 (local address 127.19.183.254:34413)
0809 12:17:15.255838 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:15.255843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:15.257117 (+ 1274us) server_negotiation.cc:407] Connection header received
0809 12:17:15.259112 (+ 1995us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:15.259120 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:15.259215 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:15.259342 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:15.260777 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:15.261844 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:15.263366 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:15.264041 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:15.264172 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:15.267874 (+ 3702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:15.267878 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:15.267882 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:15.267914 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:15.271342 (+ 3428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:15.272071 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:15.272080 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:15.272086 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:15.272256 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:15.272691 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:15.272699 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:15.272705 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:15.273114 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:15.273296 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:15.273925 (+ 629us) server_negotiation.cc:299] Negotiation successful
0809 12:17:15.274166 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":264,"threads_started":1}
I20260809 12:17:15.275206 29392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:15.256053 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.19.183.194:59397)
0809 12:17:15.256922 (+ 869us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:15.256991 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:15.258857 (+ 1866us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:15.259540 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:15.259548 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:15.260059 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:15.260597 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:15.260615 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:15.261981 (+ 1366us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:15.261984 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:15.263228 (+ 1244us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:15.263240 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:15.264224 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:15.264229 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:15.264400 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:15.265066 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:15.265087 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:15.267670 (+ 2583us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:15.271504 (+ 3834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:15.271510 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:15.271528 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:15.271917 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:15.272362 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:15.272365 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:15.272367 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:15.272561 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:15.273251 (+ 690us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:15.273256 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:15.273559 (+ 303us) client_negotiation.cc:769] Sending connection context
0809 12:17:15.273769 (+ 210us) client_negotiation.cc:240] Negotiation successful
0809 12:17:15.274126 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":549,"thread_start_us":337,"threads_started":1}
I20260809 12:17:15.276733 29384 heartbeater.cc:347] Connected to a master server at 127.19.183.254:34413
I20260809 12:17:15.277069 29384 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:15.278021 29384 heartbeater.cc:511] Master 127.19.183.254:34413 requested a full tablet report, sending...
I20260809 12:17:15.279688 28920 ts_manager.cc:194] Registered new tserver with Master: 2e535068581c4dbcaca1b42f72e5b911 (127.19.183.194:42409)
I20260809 12:17:15.280489 28920 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:59397
WARNING: no policy specified for oryx/127.19.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.195.keytab.
Entry for principal oryx/127.19.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.195.keytab.
I20260809 12:17:15.317754 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:34413
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.195.keytab
--principal=oryx/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:42821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:17:15.655049 29398 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:15.655326 29398 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:15.655530 29398 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:15.662690 29398 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:15.662847 29398 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:15.663020 29398 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:17:15.672586 29398 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:42821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.195.keytab
--principal=oryx/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:34413
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:15.673856 29398 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:15.675174 29398 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:15.688099 29403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:15.688155 29404 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:15.688153 29406 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:15.692106 29398 server_base.cc:1034] running on GCE node
I20260809 12:17:15.692690 29398 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:15.693622 29398 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:15.694803 29398 hybrid_clock.cc:648] HybridClock initialized: now 1786277835694753 us; error 35 us; skew 500 ppm
I20260809 12:17:15.695115 29398 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:15 dist-test-slave-39l8 krb5kdc[28862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277835, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:15.698196 29398 init.cc:377] Logged in from keytab as oryx/127.19.183.195@KRBTEST.COM (short username oryx)
I20260809 12:17:15.699738 29398 webserver.cc:460] Webserver started at http://127.19.183.195:42139/ using document root <none> and password file <none>
I20260809 12:17:15.700445 29398 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:15.700588 29398 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:15.700896 29398 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:15.703775 29398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "565f367d64c347598f718368daa08a26"
format_stamp: "Formatted at 2026-08-09 12:17:15 on dist-test-slave-39l8"
server_key: "f451d35b316d6da01e8c351518f0655b"
server_key_iv: "de04fe442763e20fe9b66974630c5b05"
server_key_version: "encryptionkey@0"
I20260809 12:17:15.704571 29398 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "565f367d64c347598f718368daa08a26"
format_stamp: "Formatted at 2026-08-09 12:17:15 on dist-test-slave-39l8"
server_key: "f451d35b316d6da01e8c351518f0655b"
server_key_iv: "de04fe442763e20fe9b66974630c5b05"
server_key_version: "encryptionkey@0"
I20260809 12:17:15.709523 29398 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 12:17:15.712877 29422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:15.714174 29398 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 12:17:15.714341 29398 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "565f367d64c347598f718368daa08a26"
format_stamp: "Formatted at 2026-08-09 12:17:15 on dist-test-slave-39l8"
server_key: "f451d35b316d6da01e8c351518f0655b"
server_key_iv: "de04fe442763e20fe9b66974630c5b05"
server_key_version: "encryptionkey@0"
I20260809 12:17:15.714545 29398 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:15.742074 29398 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:15.745668 29398 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:15.745903 29398 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:15.746800 29398 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:15.748123 29398 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:15.748193 29398 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:15.748272 29398 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:15.748315 29398 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:15.754086 29193 heartbeater.cc:503] Master 127.19.183.254:34413 was elected leader, sending a full tablet report...
I20260809 12:17:15.783246 29398 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:36023
I20260809 12:17:15.783299 29585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:36023 every 8 connection(s)
I20260809 12:17:15.784578 29398 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:17:15.792518 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 29398
I20260809 12:17:15.792709 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:17:15.793196 20191 external_mini_cluster.cc:1428] Setting key de7bf9711b47478a34a61f3f32da4f71
Aug 09 12:17:15 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277835, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM
I20260809 12:17:15.805887 29591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:15.787314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:39509 (local address 127.19.183.254:34413)
0809 12:17:15.787775 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:15.787780 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:15.788683 (+ 903us) server_negotiation.cc:407] Connection header received
0809 12:17:15.790557 (+ 1874us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:15.790563 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:15.790654 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:15.790808 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:15.792435 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:15.793346 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:15.795542 (+ 2196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:15.796180 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:15.796318 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:15.799710 (+ 3392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:15.799714 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:15.799717 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:15.799749 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:15.803133 (+ 3384us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:15.803973 (+ 840us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:15.803977 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:15.803979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:15.804113 (+ 134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:15.804475 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:15.804478 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:15.804480 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:15.804730 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:15.804888 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:15.805462 (+ 574us) server_negotiation.cc:299] Negotiation successful
0809 12:17:15.805625 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":224,"threads_started":1}
I20260809 12:17:15.806936 29592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:15.787839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.19.183.195:39509)
0809 12:17:15.788519 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:15.788568 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:15.790281 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:15.791042 (+ 761us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:15.791050 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:15.791531 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:15.792229 (+ 698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:15.792258 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:15.793511 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:15.793517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:15.795392 (+ 1875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:15.795409 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:15.796425 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:15.796434 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:15.796660 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:15.797321 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:15.797343 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:15.799501 (+ 2158us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:15.803283 (+ 3782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:15.803296 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:15.803313 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:15.803866 (+ 553us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:15.804210 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:15.804213 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:15.804216 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:15.804385 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:15.804851 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:15.804860 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:15.805155 (+ 295us) client_negotiation.cc:769] Sending connection context
0809 12:17:15.805376 (+ 221us) client_negotiation.cc:240] Negotiation successful
0809 12:17:15.805739 (+ 363us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"thread_start_us":268,"threads_started":1}
I20260809 12:17:15.808339 29589 heartbeater.cc:347] Connected to a master server at 127.19.183.254:34413
I20260809 12:17:15.808717 29589 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:15.809484 29589 heartbeater.cc:511] Master 127.19.183.254:34413 requested a full tablet report, sending...
I20260809 12:17:15.810945 28920 ts_manager.cc:194] Registered new tserver with Master: 565f367d64c347598f718368daa08a26 (127.19.183.195:36023)
I20260809 12:17:15.811719 28920 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:39509
I20260809 12:17:15.820937 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.19.183.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.183.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.183.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.253.keytab.
Entry for principal oryx/127.19.183.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.253.keytab.
I20260809 12:17:15.876183 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.253:40421
--webserver_interface=127.19.183.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.253.keytab
--principal=oryx/127.19.183.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:42821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:34413,127.19.183.253:40421
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:17:16.216964 29596 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:16.217227 29596 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:16.217396 29596 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:16.224673 29596 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:17:16.224804 29596 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:16.224900 29596 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:16.224992 29596 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:17:16.225067 29596 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:17:16.235467 29596 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:42821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:34413,127.19.183.253:40421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.253.keytab
--principal=oryx/127.19.183.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.253:40421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.183.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
W20260809 12:17:16.235841 29596 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.19.183.254:34413,127.19.183.253:40421'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 12:17:16.236891 29596 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:16.238200 29596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:16.250196 29607 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:16.250166 29608 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:16.250260 29610 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:16.256057 29596 server_base.cc:1034] running on GCE node
I20260809 12:17:16.257108 29596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:16.258754 29596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:16.259985 29596 hybrid_clock.cc:648] HybridClock initialized: now 1786277836259909 us; error 67 us; skew 500 ppm
I20260809 12:17:16.260337 29596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:16 dist-test-slave-39l8 krb5kdc[28862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277836, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:16.264386 29596 init.cc:377] Logged in from keytab as oryx/127.19.183.253@KRBTEST.COM (short username oryx)
I20260809 12:17:16.266083 29596 webserver.cc:460] Webserver started at http://127.19.183.253:32989/ using document root <none> and password file <none>
I20260809 12:17:16.266974 29596 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:16.267086 29596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:16.267395 29596 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:16.270885 29596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/data/instance:
uuid: "b399be0c91ba480a873d964b624c404e"
format_stamp: "Formatted at 2026-08-09 12:17:16 on dist-test-slave-39l8"
server_key: "d95f08bdd72915bc95a911bf87c39f86"
server_key_iv: "b2ea4ceb33a98b5d23ac671580e6c5b8"
server_key_version: "encryptionkey@0"
I20260809 12:17:16.271739 29596 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/wal/instance:
uuid: "b399be0c91ba480a873d964b624c404e"
format_stamp: "Formatted at 2026-08-09 12:17:16 on dist-test-slave-39l8"
server_key: "d95f08bdd72915bc95a911bf87c39f86"
server_key_iv: "b2ea4ceb33a98b5d23ac671580e6c5b8"
server_key_version: "encryptionkey@0"
I20260809 12:17:16.277243 29596 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:17:16.280874 29620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:16.282152 29596 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:17:16.282346 29596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/wal
uuid: "b399be0c91ba480a873d964b624c404e"
format_stamp: "Formatted at 2026-08-09 12:17:16 on dist-test-slave-39l8"
server_key: "d95f08bdd72915bc95a911bf87c39f86"
server_key_iv: "b2ea4ceb33a98b5d23ac671580e6c5b8"
server_key_version: "encryptionkey@0"
I20260809 12:17:16.282573 29596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:16.284000 29384 heartbeater.cc:503] Master 127.19.183.254:34413 was elected leader, sending a full tablet report...
I20260809 12:17:16.301031 29596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:16.304831 29596 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:16.305053 29596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:16.326499 29596 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.253:40421
I20260809 12:17:16.326506 29695 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.253:40421 every 8 connection(s)
I20260809 12:17:16.327921 29596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/data/info.pb
I20260809 12:17:16.329185 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 29596
I20260809 12:17:16.332482 29696 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:16.343067 29696 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:17:16 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.183.253@KRBTEST.COM
Aug 09 12:17:16 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277836, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.253@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM
I20260809 12:17:16.356468 29700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:16.331316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50440 (local address 127.19.183.253:40421)
0809 12:17:16.332160 (+ 844us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:16.332175 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:16.332225 (+ 50us) server_negotiation.cc:407] Connection header received
0809 12:17:16.334871 (+ 2646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:16.334900 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:16.335355 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:16.336114 (+ 759us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:16.337490 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.339919 (+ 2429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.341207 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.342385 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.342562 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:16.347836 (+ 5274us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:16.347849 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:16.347868 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:16.347907 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:16.351895 (+ 3988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:16.352548 (+ 653us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:16.352556 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:16.352565 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:16.352768 (+ 203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:16.353145 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:16.353155 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:16.353158 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:16.353722 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:16.354089 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:16.354520 (+ 431us) server_negotiation.cc:299] Negotiation successful
0809 12:17:16.355003 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":360,"threads_started":1}
I20260809 12:17:16.363324 29701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:16.346034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43228 (local address 127.19.183.254:34413)
0809 12:17:16.346817 (+ 783us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:16.346822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:16.347824 (+ 1002us) server_negotiation.cc:407] Connection header received
0809 12:17:16.348035 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:16.348040 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:16.348132 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:16.348269 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:16.350163 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.351260 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.353089 (+ 1829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.353482 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.353635 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:16.357264 (+ 3629us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:16.357269 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:16.357272 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:16.357312 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:16.360398 (+ 3086us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:16.361460 (+ 1062us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:16.361463 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:16.361465 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:16.361584 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:16.361950 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:16.361953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:16.361954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:16.362232 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:16.362334 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:16.362885 (+ 551us) server_negotiation.cc:299] Negotiation successful
0809 12:17:16.363048 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":630,"thread_start_us":502,"threads_started":1}
I20260809 12:17:16.363695 29702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:16.346296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.0.0.1:43228)
0809 12:17:16.347485 (+ 1189us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:16.347555 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:16.347893 (+ 338us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:16.348362 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:16.348371 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:16.349416 (+ 1045us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:16.349943 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:16.349968 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.351403 (+ 1435us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.351414 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:16.352936 (+ 1522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:16.352947 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.353687 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.353695 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:16.353868 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:16.354477 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:16.354501 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:16.357047 (+ 2546us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:16.360548 (+ 3501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:16.360561 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:16.360581 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:16.361348 (+ 767us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:16.361686 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:16.361693 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:16.361698 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:16.361853 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:16.362356 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:16.362366 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:16.362556 (+ 190us) client_negotiation.cc:769] Sending connection context
0809 12:17:16.362799 (+ 243us) client_negotiation.cc:240] Negotiation successful
0809 12:17:16.363140 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":953,"thread_start_us":812,"threads_started":1}
I20260809 12:17:16.365640 29696 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:17:16 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277836, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.253@KRBTEST.COM for oryx/127.19.183.253@KRBTEST.COM
I20260809 12:17:16.379436 29702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:16.366563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:40421 (local address 127.0.0.1:50452)
0809 12:17:16.366788 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:16.366803 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:16.366918 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:16.367220 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:16.367224 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:16.367689 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:16.367848 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:16.367856 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.368823 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.368832 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:16.369867 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:16.369878 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.370645 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.370652 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:16.370837 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:16.371335 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:16.371351 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:16.373546 (+ 2195us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:16.376631 (+ 3085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:16.376636 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:16.376640 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:16.377121 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:16.377618 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:16.377622 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:16.377626 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:16.377776 (+ 150us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:16.378327 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:16.378332 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:16.378463 (+ 131us) client_negotiation.cc:769] Sending connection context
0809 12:17:16.378614 (+ 151us) client_negotiation.cc:240] Negotiation successful
0809 12:17:16.378940 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 12:17:16.379667 29700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:16.366377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50452 (local address 127.19.183.253:40421)
0809 12:17:16.366523 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:16.366530 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:16.366865 (+ 335us) server_negotiation.cc:407] Connection header received
0809 12:17:16.367024 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:16.367029 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:16.367113 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:16.367200 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:16.367980 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.368677 (+ 697us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.370023 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.370476 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.370601 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:16.373731 (+ 3130us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:16.373736 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:16.373739 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:16.373764 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:16.376479 (+ 2715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:16.377236 (+ 757us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:16.377243 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:16.377246 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:16.377487 (+ 241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:16.377901 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:16.377907 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:16.377910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:16.378202 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:16.378344 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:16.378745 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 12:17:16.379139 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":126,"server-negotiator.queue_time_us":44}
I20260809 12:17:16.385236 29696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e: Bootstrap starting.
I20260809 12:17:16.388814 29696 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:16.389815 29696 log.cc:824] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e: Log is configured to *not* fsync() on all Append() calls
I20260809 12:17:16.392827 29696 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e: No bootstrap required, opened a new log
I20260809 12:17:16.400499 29696 raft_consensus.cc:348] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } }
I20260809 12:17:16.400807 29696 raft_consensus.cc:374] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:16.400882 29696 raft_consensus.cc:729] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b399be0c91ba480a873d964b624c404e, State: Initialized, Role: FOLLOWER
I20260809 12:17:16.401401 29696 consensus_queue.cc:260] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } }
I20260809 12:17:16.402420 29705 sys_catalog.cc:455] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } } }
I20260809 12:17:16.402837 29705 sys_catalog.cc:458] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:17:16.403287 29696 sys_catalog.cc:564] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [sys.catalog]: configured and running, proceeding with master startup.
W20260809 12:17:16.418392 29726 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 12:17:16.418537 29726 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 12:17:16.419902 29596 master_runner.cc:428] Detected that this master b399be0c91ba480a873d964b624c404e is joining an existing cluster
I20260809 12:17:16.420017 29596 master_runner.cc:432] Initiating AddMaster RPC to add 127.19.183.253:40421
WARNING: no policy specified for oryx/127.19.183.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.183.252@KRBTEST.COM" created.
I20260809 12:17:16.454373 29701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:16.425488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43230 (local address 127.19.183.254:34413)
0809 12:17:16.425914 (+ 426us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:16.425922 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:16.426888 (+ 966us) server_negotiation.cc:407] Connection header received
0809 12:17:16.431807 (+ 4919us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:16.431816 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:16.431970 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:16.432134 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:16.435071 (+ 2937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.436316 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.443559 (+ 7243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.444274 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.444403 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:16.447095 (+ 2692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:16.447104 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:16.447108 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:16.447159 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:16.450984 (+ 3825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:16.451854 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:16.451861 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:16.451867 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:16.452056 (+ 189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:16.452504 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:16.452510 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:16.452515 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:16.452926 (+ 411us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:16.453087 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:16.453676 (+ 589us) server_negotiation.cc:299] Negotiation successful
0809 12:17:16.453944 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280}
I20260809 12:17:16.454401 29738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:16.425455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.0.0.1:43230)
0809 12:17:16.426764 (+ 1309us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:16.426789 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:16.431408 (+ 4619us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:16.432371 (+ 963us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:16.432378 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:16.434265 (+ 1887us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:16.434886 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:16.434905 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.436581 (+ 1676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.436589 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:16.443391 (+ 6802us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:16.443413 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.444704 (+ 1291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.444712 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:16.444846 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:16.445497 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:16.445516 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:16.446827 (+ 1311us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:16.451145 (+ 4318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:16.451215 (+ 70us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:16.451221 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:16.451701 (+ 480us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:16.452188 (+ 487us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:16.452194 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:16.452203 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:16.452373 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:16.453079 (+ 706us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:16.453089 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:16.453323 (+ 234us) client_negotiation.cc:769] Sending connection context
0809 12:17:16.453531 (+ 208us) client_negotiation.cc:240] Negotiation successful
0809 12:17:16.453852 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1086,"mutex_wait_us":68,"thread_start_us":864,"threads_started":1}
I20260809 12:17:16.461793 29739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:16.426217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:40421 (local address 127.0.0.1:50464)
0809 12:17:16.427134 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:16.427545 (+ 411us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:16.427717 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:16.428203 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:16.428210 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:16.428983 (+ 773us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:16.429399 (+ 416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:16.429417 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.431326 (+ 1909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.431334 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:16.439642 (+ 8308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:16.439662 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.443098 (+ 3436us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.443110 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:16.443450 (+ 340us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:16.444248 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:16.444273 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:16.446005 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:16.457686 (+ 11681us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:16.457690 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:16.457694 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:16.458289 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:16.459198 (+ 909us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:16.459206 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:16.459212 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:16.459589 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:16.460340 (+ 751us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:16.460347 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:16.460503 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 12:17:16.460648 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 12:17:16.460973 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":773,"mutex_wait_us":24,"thread_start_us":533,"threads_started":1}
I20260809 12:17:16.463712 29700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:16.425377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50464 (local address 127.19.183.253:40421)
0809 12:17:16.425597 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:16.425603 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:16.427618 (+ 2015us) server_negotiation.cc:407] Connection header received
0809 12:17:16.427846 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:16.427853 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:16.427929 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:16.427996 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:16.429567 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.430795 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.439963 (+ 9168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.440927 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.441078 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:16.446199 (+ 5121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:16.446206 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:16.446211 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:16.446255 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:16.457530 (+ 11275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:16.458452 (+ 922us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:16.458460 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:16.458464 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:16.458661 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:16.459713 (+ 1052us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:16.459720 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:16.459725 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:16.460120 (+ 395us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:16.460275 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:16.462164 (+ 1889us) server_negotiation.cc:299] Negotiation successful
0809 12:17:16.462462 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 12:17:16 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277834, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.254@KRBTEST.COM for oryx/127.19.183.253@KRBTEST.COM
I20260809 12:17:16.496026 29742 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:16.462377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:40421 (local address 127.0.0.1:50472)
0809 12:17:16.467530 (+ 5153us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:16.467574 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:16.467791 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:16.470874 (+ 3083us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:16.470885 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:16.472029 (+ 1144us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:16.472684 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:16.472700 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.475450 (+ 2750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.475455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:16.477693 (+ 2238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:16.477710 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.479475 (+ 1765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.479483 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:16.479782 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:16.480793 (+ 1011us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:16.480818 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:16.484784 (+ 3966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:16.491912 (+ 7128us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:16.491926 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:16.491935 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:16.492788 (+ 853us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:16.493456 (+ 668us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:16.493461 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:16.493465 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:16.493679 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:16.494413 (+ 734us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:16.494424 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:16.494745 (+ 321us) client_negotiation.cc:769] Sending connection context
0809 12:17:16.495037 (+ 292us) client_negotiation.cc:240] Negotiation successful
0809 12:17:16.495446 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4861,"thread_start_us":4690,"threads_started":1}
I20260809 12:17:16.496081 29743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:16.462931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50472 (local address 127.19.183.253:40421)
0809 12:17:16.463889 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:16.463896 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:16.467667 (+ 3771us) server_negotiation.cc:407] Connection header received
0809 12:17:16.468038 (+ 371us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:16.468046 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:16.468169 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:16.468275 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:16.472985 (+ 4710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.474691 (+ 1706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.477882 (+ 3191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.478904 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.479027 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:16.485216 (+ 6189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:16.485227 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:16.485236 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:16.485287 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:16.491447 (+ 6160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:16.492923 (+ 1476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:16.492928 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:16.492930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:16.493298 (+ 368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:16.493875 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:16.493881 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:16.493884 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:16.494226 (+ 342us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:16.494335 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:16.495142 (+ 807us) server_negotiation.cc:299] Negotiation successful
0809 12:17:16.495420 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":802,"thread_start_us":623,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.183.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.252.keytab.
Entry for principal oryx/127.19.183.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.252.keytab.
I20260809 12:17:16.499150 28920 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.19.183.253:40421: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b399be0c91ba480a873d964b624c404e"
member_type: NON_VOTER
last_known_addr {
host: "127.19.183.253"
port: 40421
}
attrs {
promote: true
}
}
dest_uuid: "7852578af3644424bdcd20194612fdac"
cas_config_opid_index: -1
I20260809 12:17:16.501039 28920 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: NON_VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: true } }
I20260809 12:17:16.503199 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.252:41811
--webserver_interface=127.19.183.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.252.keytab
--principal=oryx/127.19.183.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:42821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:34413,127.19.183.253:40421,127.19.183.252:41811
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:17:16.517326 29663 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:16.519923 29750 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER b399be0c91ba480a873d964b624c404e (127.19.183.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: NON_VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: true } } }
I20260809 12:17:16.523736 29663 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [term 1 FOLLOWER]: Refusing update from remote peer 7852578af3644424bdcd20194612fdac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 12:17:16.527182 29750 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.19.183.253:40421
I20260809 12:17:16.527976 29750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7852578af3644424bdcd20194612fdac" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: NON_VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: true } } }
I20260809 12:17:16.528342 29750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:16.526913 29749 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [LEADER]: Connected to new peer: Peer: permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: NON_VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:17:16.530179 29596 master.cc:537] Master@127.19.183.253:40421 shutting down...
W20260809 12:17:16.538048 28900 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 12:17:16.542932 28900 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac -> Peer b399be0c91ba480a873d964b624c404e (127.19.183.253:40421): Couldn't send request to peer b399be0c91ba480a873d964b624c404e. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 12:17:16.547137 29596 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:17:16.547592 29596 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:17:16.547791 29596 tablet_replica.cc:330] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e: stopping tablet replica
I20260809 12:17:16.555567 29596 master.cc:552] Master@127.19.183.253:40421 shutdown complete.
I20260809 12:17:16.555692 29596 master_runner.cc:305] Clearing existing system tablet
I20260809 12:17:16.556176 29596 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:16.556281 29596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:16.559376 29596 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.004s
I20260809 12:17:16.561026 29761 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:16.561372 29596 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:16.561669 29596 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260809 12:17:16.561757 29596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/wal
uuid: "b399be0c91ba480a873d964b624c404e"
format_stamp: "Formatted at 2026-08-09 12:17:16 on dist-test-slave-39l8"
server_key: "d95f08bdd72915bc95a911bf87c39f86"
server_key_iv: "b2ea4ceb33a98b5d23ac671580e6c5b8"
server_key_version: "encryptionkey@0"
I20260809 12:17:16.567070 29596 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:17:16.574065 29596 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 12:17:16.574340 29596 log.cc:1196] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 12:17:16.575102 29596 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e: Deleting consensus metadata
I20260809 12:17:16.575460 29596 master_runner.cc:315] Copying system tablet from 127.19.183.254:34413
I20260809 12:17:16.578866 29596 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e: tablet copy: Beginning tablet copy session from remote peer at address 127.19.183.254:34413
I20260809 12:17:16.595746 29771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:16.579967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.0.0.1:43246)
0809 12:17:16.580531 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:16.580558 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:16.580719 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:16.581215 (+ 496us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:16.581219 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:16.581765 (+ 546us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:16.582032 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:16.582045 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.583778 (+ 1733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.583783 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:16.585396 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:16.585412 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.586768 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.586774 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:16.586954 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:16.587627 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:16.587651 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:16.588687 (+ 1036us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:16.592866 (+ 4179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:16.592877 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:16.592881 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:16.593401 (+ 520us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:16.593821 (+ 420us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:16.593827 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:16.593831 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:16.594008 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:16.594613 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:16.594625 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:16.594832 (+ 207us) client_negotiation.cc:769] Sending connection context
0809 12:17:16.595006 (+ 174us) client_negotiation.cc:240] Negotiation successful
0809 12:17:16.595303 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"thread_start_us":296,"threads_started":1}
I20260809 12:17:16.596681 29701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:16.579933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43246 (local address 127.19.183.254:34413)
0809 12:17:16.580175 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:16.580184 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:16.580672 (+ 488us) server_negotiation.cc:407] Connection header received
0809 12:17:16.580884 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:16.580891 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:16.581034 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:16.581169 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:16.582215 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.583597 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.585599 (+ 2002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:16.586575 (+ 976us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:16.586742 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:16.588911 (+ 2169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:16.588917 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:16.588921 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:16.588958 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:16.592725 (+ 3767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:16.593533 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:16.593538 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:16.593541 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:16.593700 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:16.594127 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:16.594131 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:16.594134 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:16.594491 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:16.594638 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:16.596099 (+ 1461us) server_negotiation.cc:299] Negotiation successful
0809 12:17:16.596314 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 12:17:16.597595 28945 tablet_copy_service.cc:140] P 7852578af3644424bdcd20194612fdac: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b399be0c91ba480a873d964b624c404e ({username='oryx', principal='oryx/127.19.183.253@KRBTEST.COM'} at 127.0.0.1:43246)
I20260809 12:17:16.598037 28945 tablet_copy_service.cc:161] P 7852578af3644424bdcd20194612fdac: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b399be0c91ba480a873d964b624c404e at {username='oryx', principal='oryx/127.19.183.253@KRBTEST.COM'} at 127.0.0.1:43246: session id = b399be0c91ba480a873d964b624c404e-00000000000000000000000000000000
I20260809 12:17:16.602860 28945 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 12:17:16.605773 29596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 12:17:16.612867 29596 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e: tablet copy: Starting download of 0 data blocks...
I20260809 12:17:16.613313 29596 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e: tablet copy: Starting download of 1 WAL segments...
I20260809 12:17:16.617282 29596 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 12:17:16.621968 28945 tablet_copy_service.cc:342] P 7852578af3644424bdcd20194612fdac: Request end of tablet copy session b399be0c91ba480a873d964b624c404e-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.19.183.253@KRBTEST.COM'} at 127.0.0.1:43246
I20260809 12:17:16.622224 28945 tablet_copy_service.cc:433] P 7852578af3644424bdcd20194612fdac: ending tablet copy session b399be0c91ba480a873d964b624c404e-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b399be0c91ba480a873d964b624c404e
I20260809 12:17:16.635241 29596 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:16.637996 29772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:16.638418 29773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:16.645318 29778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:16.646142 29596 server_base.cc:1034] running on GCE node
I20260809 12:17:16.646476 29596 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:16.647140 29596 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:16.648350 29596 hybrid_clock.cc:648] HybridClock initialized: now 1786277836648270 us; error 49 us; skew 500 ppm
I20260809 12:17:16.648598 29596 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:16 dist-test-slave-39l8 krb5kdc[28862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277836, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:16.650933 29596 init.cc:377] Logged in from keytab as oryx/127.19.183.253@KRBTEST.COM (short username oryx)
I20260809 12:17:16.652170 29596 webserver.cc:460] Webserver started at http://127.19.183.253:35995/ using document root <none> and password file <none>
I20260809 12:17:16.652558 29596 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:16.652642 29596 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:16.655821 29596 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.000s sys 0.003s
I20260809 12:17:16.658022 29790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:16.658495 29596 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260809 12:17:16.658656 29596 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/wal
uuid: "b399be0c91ba480a873d964b624c404e"
format_stamp: "Formatted at 2026-08-09 12:17:16 on dist-test-slave-39l8"
server_key: "d95f08bdd72915bc95a911bf87c39f86"
server_key_iv: "b2ea4ceb33a98b5d23ac671580e6c5b8"
server_key_version: "encryptionkey@0"
I20260809 12:17:16.658855 29596 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:16.709132 29596 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:16.713666 29596 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:16.750663 29596 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.253:40421
I20260809 12:17:16.750697 29866 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.253:40421 every 8 connection(s)
I20260809 12:17:16.751739 29596 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-1/data/info.pb
I20260809 12:17:16.757488 29867 sys_catalog.cc:263] Verifying existing consensus state
I20260809 12:17:16.761361 29867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e: Bootstrap starting.
I20260809 12:17:16.782428 29867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:17:16.783082 29867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e: Bootstrap complete.
I20260809 12:17:16.784343 29867 raft_consensus.cc:348] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: NON_VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: true } }
I20260809 12:17:16.784682 29867 raft_consensus.cc:729] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b399be0c91ba480a873d964b624c404e, State: Initialized, Role: LEARNER
I20260809 12:17:16.784988 29867 consensus_queue.cc:260] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: NON_VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: true } }
I20260809 12:17:16.789686 29874 sys_catalog.cc:455] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: NON_VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: true } } }
I20260809 12:17:16.790047 29874 sys_catalog.cc:458] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [sys.catalog]: This master's current role is: LEARNER
I20260809 12:17:16.790094 29867 sys_catalog.cc:564] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:17:16.802098 29890 catalog_manager.cc:1259] Loaded cluster ID: e474dd64bd7a4984a89c538000646418
I20260809 12:17:16.802192 29890 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e: loading cluster ID for follower catalog manager: success
I20260809 12:17:16.805588 29890 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e: acquiring CA information for follower catalog manager: success
I20260809 12:17:16.808046 29890 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 12:17:16.814069 29589 heartbeater.cc:503] Master 127.19.183.254:34413 was elected leader, sending a full tablet report...
W20260809 12:17:16.962579 29752 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:16.963045 29752 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:16.963335 29752 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:16.971899 29752 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:17:16.972131 29752 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:16.972285 29752 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:16.972416 29752 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:17:16.972512 29752 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:17:16.983556 29752 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:42821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:34413,127.19.183.253:40421,127.19.183.252:41811
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.252.keytab
--principal=oryx/127.19.183.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.252:41811
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.183.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:16.985107 29752 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:16.986606 29752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:17.006304 29895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:17.007483 29900 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:17.011662 29896 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:17.011914 29752 server_base.cc:1034] running on GCE node
I20260809 12:17:17.013121 29752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:17.015832 29752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:17.017220 29752 hybrid_clock.cc:648] HybridClock initialized: now 1786277837017050 us; error 52 us; skew 500 ppm
I20260809 12:17:17.017865 29752 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:17 dist-test-slave-39l8 krb5kdc[28862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277837, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:17.023866 29752 init.cc:377] Logged in from keytab as oryx/127.19.183.252@KRBTEST.COM (short username oryx)
I20260809 12:17:17.026262 29752 webserver.cc:460] Webserver started at http://127.19.183.252:45417/ using document root <none> and password file <none>
I20260809 12:17:17.027274 29752 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:17.027498 29752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:17.027899 29752 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:17.031728 29752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/data/instance:
uuid: "5f59b93ddf0e4cc8be0df79193d05d8f"
format_stamp: "Formatted at 2026-08-09 12:17:17 on dist-test-slave-39l8"
server_key: "db5d4e5326f139845de32a118c231f87"
server_key_iv: "c61fd8a242ec7d1b1996db49aa368d6c"
server_key_version: "encryptionkey@0"
I20260809 12:17:17.032665 29752 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/wal/instance:
uuid: "5f59b93ddf0e4cc8be0df79193d05d8f"
format_stamp: "Formatted at 2026-08-09 12:17:17 on dist-test-slave-39l8"
server_key: "db5d4e5326f139845de32a118c231f87"
server_key_iv: "c61fd8a242ec7d1b1996db49aa368d6c"
server_key_version: "encryptionkey@0"
I20260809 12:17:17.039911 29752 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.000s sys 0.007s
I20260809 12:17:17.044597 29912 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:17.046089 29752 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.001s
I20260809 12:17:17.046353 29752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/wal
uuid: "5f59b93ddf0e4cc8be0df79193d05d8f"
format_stamp: "Formatted at 2026-08-09 12:17:17 on dist-test-slave-39l8"
server_key: "db5d4e5326f139845de32a118c231f87"
server_key_iv: "c61fd8a242ec7d1b1996db49aa368d6c"
server_key_version: "encryptionkey@0"
I20260809 12:17:17.046680 29752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:17.062183 29752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:17.067529 29752 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:17.067960 29752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:17.116818 29752 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.252:41811
I20260809 12:17:17.116972 29982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.252:41811 every 8 connection(s)
I20260809 12:17:17.118849 29752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/data/info.pb
I20260809 12:17:17.125639 29983 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:17.126359 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 29752
I20260809 12:17:17.139672 29983 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:17:17.144614 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:17.128900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:40421 (local address 127.0.0.1:50484)
0809 12:17:17.135597 (+ 6697us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:17.135623 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:17.135842 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:17.138043 (+ 2201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:17.138048 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:17.138077 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:17:17.138350 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:17.138361 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.139890 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.139895 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:17.141360 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:17.141373 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.143475 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.143484 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:17.143720 (+ 236us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:17.143785 (+ 65us) client_negotiation.cc:769] Sending connection context
0809 12:17:17.143888 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 12:17:17.144101 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6499,"thread_start_us":7699,"threads_started":1}
I20260809 12:17:17.145612 29989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:17.128344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50484 (local address 127.19.183.253:40421)
0809 12:17:17.137585 (+ 9241us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:17.137591 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:17.137614 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:17:17.137694 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:17.137699 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:17.137812 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:17.138003 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:17:17.138501 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.139726 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.141533 (+ 1807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.143245 (+ 1712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.143456 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:17.143651 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:17.144875 (+ 1224us) server_negotiation.cc:299] Negotiation successful
0809 12:17:17.145101 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9031,"thread_start_us":8846,"threads_started":1}
I20260809 12:17:17.146898 29831 raft_consensus.cc:1203] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260809 12:17:17.148523 29874 sys_catalog.cc:455] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7852578af3644424bdcd20194612fdac. Latest consensus state: current_term: 1 leader_uuid: "7852578af3644424bdcd20194612fdac" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: NON_VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: true } } }
I20260809 12:17:17.148850 29874 sys_catalog.cc:458] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [sys.catalog]: This master's current role is: LEARNER
Aug 09 12:17:17 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.183.252@KRBTEST.COM
Aug 09 12:17:17 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277837, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.252@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM
I20260809 12:17:17.164069 29994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:17.129382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54192 (local address 127.19.183.252:41811)
0809 12:17:17.138001 (+ 8619us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:17.138017 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:17.138077 (+ 60us) server_negotiation.cc:407] Connection header received
0809 12:17:17.140555 (+ 2478us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:17.140586 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:17.141165 (+ 579us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:17.141693 (+ 528us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:17.143913 (+ 2220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.146474 (+ 2561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.148368 (+ 1894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.149263 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.149391 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:17.155297 (+ 5906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:17.155313 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:17.155330 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:17.155399 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:17.159172 (+ 3773us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:17.160104 (+ 932us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:17.160111 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:17.160117 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:17.160277 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:17.160683 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:17.160686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:17.160688 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:17.161228 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:17.161573 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:17.162091 (+ 518us) server_negotiation.cc:299] Negotiation successful
0809 12:17:17.162482 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8350,"thread_start_us":8052,"threads_started":1}
I20260809 12:17:17.164207 29998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:17.144617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.0.0.1:43250)
0809 12:17:17.145903 (+ 1286us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:17.145954 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:17.146164 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:17.146907 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:17.146911 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:17.148000 (+ 1089us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:17.148499 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:17.148511 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.150048 (+ 1537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.150055 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:17.151849 (+ 1794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:17.151861 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.153123 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.153129 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:17.153286 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:17.153911 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:17.153944 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:17.156946 (+ 3002us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:17.161062 (+ 4116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:17.161075 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:17.161082 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:17.161838 (+ 756us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:17.162278 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:17.162283 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:17.162286 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:17.162489 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:17.163100 (+ 611us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:17.163106 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:17.163257 (+ 151us) client_negotiation.cc:769] Sending connection context
0809 12:17:17.163425 (+ 168us) client_negotiation.cc:240] Negotiation successful
0809 12:17:17.163639 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":791,"thread_start_us":548,"threads_started":1}
I20260809 12:17:17.164239 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:17.143802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43250 (local address 127.19.183.254:34413)
0809 12:17:17.144553 (+ 751us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:17.144559 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:17.146108 (+ 1549us) server_negotiation.cc:407] Connection header received
0809 12:17:17.146305 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:17.146311 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:17.146388 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:17.146524 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:17.148667 (+ 2143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.149868 (+ 1201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.151999 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.152876 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.153030 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:17.157137 (+ 4107us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:17.157146 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:17.157149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:17.157179 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:17.160932 (+ 3753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:17.161970 (+ 1038us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:17.161975 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:17.161978 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:17.162135 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:17.162618 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:17.162623 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:17.162626 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:17.162978 (+ 352us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:17.163141 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:17.163572 (+ 431us) server_negotiation.cc:299] Negotiation successful
0809 12:17:17.163805 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":600,"thread_start_us":445,"threads_started":1}
I20260809 12:17:17.166445 29983 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:17:17 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277837, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.252@KRBTEST.COM for oryx/127.19.183.253@KRBTEST.COM
I20260809 12:17:17.180243 29998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:17.167413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:40421 (local address 127.0.0.1:50500)
0809 12:17:17.167639 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:17.167660 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:17.167789 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:17.168093 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:17.168098 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:17.168481 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:17.168631 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:17.168639 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.169703 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.169708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:17.170801 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:17.170814 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.171581 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.171586 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:17.171810 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:17.172463 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:17.172479 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:17.174588 (+ 2109us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:17.177694 (+ 3106us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:17.177700 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:17.177704 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:17.178115 (+ 411us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:17.178512 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:17.178517 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:17.178521 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:17.178685 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:17.179212 (+ 527us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:17.179217 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:17.179355 (+ 138us) client_negotiation.cc:769] Sending connection context
0809 12:17:17.179489 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 12:17:17.179805 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 12:17:17.180336 29989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:17.167446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50500 (local address 127.19.183.253:40421)
0809 12:17:17.167640 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:17.167647 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:17.167731 (+ 84us) server_negotiation.cc:407] Connection header received
0809 12:17:17.167899 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:17.167905 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:17.167972 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:17.168071 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:17.168771 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.169533 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.170932 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.171445 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.171574 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:17.174788 (+ 3214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:17.174793 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:17.174796 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:17.174826 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:17.177561 (+ 2735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:17.178236 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:17.178241 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:17.178244 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:17.178398 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:17.178811 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:17.178816 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:17.178819 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:17.179088 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:17.179243 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:17.179576 (+ 333us) server_negotiation.cc:299] Negotiation successful
0809 12:17:17.179805 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 12:17:17.181535 29983 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:17:17 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277837, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.252@KRBTEST.COM for oryx/127.19.183.252@KRBTEST.COM
I20260809 12:17:17.195042 29998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:17.182339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:41811 (local address 127.0.0.1:54200)
0809 12:17:17.182454 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:17.182468 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:17.182583 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:17.182898 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:17.182902 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:17.183300 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:17.183444 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:17.183452 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.184615 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.184622 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:17.185611 (+ 989us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:17.185622 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.186341 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.186346 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:17.186607 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:17.187181 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:17.187197 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:17.189230 (+ 2033us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:17.192397 (+ 3167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:17.192403 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:17.192406 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:17.192791 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:17.193152 (+ 361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:17.193157 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:17.193161 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:17.193365 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:17.193946 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:17.193951 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:17.194131 (+ 180us) client_negotiation.cc:769] Sending connection context
0809 12:17:17.194283 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 12:17:17.194534 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 12:17:17.195051 29994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:17.182229 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54200 (local address 127.19.183.252:41811)
0809 12:17:17.182381 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:17.182391 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:17.182530 (+ 139us) server_negotiation.cc:407] Connection header received
0809 12:17:17.182725 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:17.182731 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:17.182789 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:17.182880 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:17.183572 (+ 692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.184486 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.185742 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.186197 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.186334 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:17.189419 (+ 3085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:17.189423 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:17.189426 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:17.189452 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:17.192276 (+ 2824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:17.192901 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:17.192908 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:17.192911 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:17.193032 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:17.193493 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:17.193499 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:17.193516 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:17.193837 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:17.193977 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:17.194381 (+ 404us) server_negotiation.cc:299] Negotiation successful
0809 12:17:17.194691 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":115,"server-negotiator.queue_time_us":54}
I20260809 12:17:17.200453 29983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f: Bootstrap starting.
I20260809 12:17:17.204212 29983 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:17.205509 29983 log.cc:824] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f: Log is configured to *not* fsync() on all Append() calls
I20260809 12:17:17.208382 29983 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f: No bootstrap required, opened a new log
I20260809 12:17:17.217082 29983 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } }
I20260809 12:17:17.217557 29983 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:17.217754 29983 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f59b93ddf0e4cc8be0df79193d05d8f, State: Initialized, Role: FOLLOWER
I20260809 12:17:17.218403 29983 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } }
I20260809 12:17:17.219449 30000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } } }
I20260809 12:17:17.219869 30000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:17:17.220537 29983 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [sys.catalog]: configured and running, proceeding with master startup.
W20260809 12:17:17.232486 30021 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 12:17:17.232663 30021 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 12:17:17.236523 29752 master_runner.cc:428] Detected that this master 5f59b93ddf0e4cc8be0df79193d05d8f is joining an existing cluster
I20260809 12:17:17.236727 29752 master_runner.cc:432] Initiating AddMaster RPC to add 127.19.183.252:41811
I20260809 12:17:17.270967 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 29004
I20260809 12:17:17.292256 30030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:17.246429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:41811 (local address 127.0.0.1:54202)
0809 12:17:17.259888 (+ 13459us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:17.259913 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:17.260333 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:17.261006 (+ 673us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:17.261013 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:17.263081 (+ 2068us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:17.268697 (+ 5616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:17.268718 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.270865 (+ 2147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.270871 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:17.273487 (+ 2616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:17.273502 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.275279 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.275288 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:17.275500 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:17.276405 (+ 905us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:17.276428 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:17.278189 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:17.285732 (+ 7543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:17.285737 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:17.285740 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:17.286504 (+ 764us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:17.288730 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:17.288742 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:17.288746 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:17.288982 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:17.289808 (+ 826us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:17.289813 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:17.290051 (+ 238us) client_negotiation.cc:769] Sending connection context
0809 12:17:17.290269 (+ 218us) client_negotiation.cc:240] Negotiation successful
0809 12:17:17.290609 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13291,"thread_start_us":20360,"threads_started":1}
I20260809 12:17:17.293447 29994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:17.241897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54202 (local address 127.19.183.252:41811)
0809 12:17:17.242392 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:17.242398 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:17.260203 (+ 17805us) server_negotiation.cc:407] Connection header received
0809 12:17:17.260480 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:17.260489 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:17.260598 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:17.260690 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:17.268862 (+ 8172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.270439 (+ 1577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.273664 (+ 3225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.274587 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.274763 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:17.278382 (+ 3619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:17.278388 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:17.278393 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:17.278433 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:17.285557 (+ 7124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:17.286647 (+ 1090us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:17.286651 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:17.286654 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:17.286884 (+ 230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:17.289128 (+ 2244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:17.289136 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:17.289145 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:17.289579 (+ 434us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:17.289742 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:17.291246 (+ 1504us) server_negotiation.cc:299] Negotiation successful
0809 12:17:17.291614 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 12:17:17.293678 29989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:17.241897 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50502 (local address 127.19.183.253:40421)
0809 12:17:17.242088 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:17.242094 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:17.268802 (+ 26708us) server_negotiation.cc:407] Connection header received
0809 12:17:17.269133 (+ 331us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:17.269139 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:17.269233 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:17.269309 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:17.272755 (+ 3446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.274170 (+ 1415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.277742 (+ 3572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.278861 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.278987 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:17.284135 (+ 5148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:17.284143 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:17.284148 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:17.284191 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:17.288820 (+ 4629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:17.289730 (+ 910us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:17.289735 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:17.289738 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:17.289994 (+ 256us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:17.290580 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:17.290585 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:17.290588 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:17.291120 (+ 532us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:17.291250 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:17.292765 (+ 1515us) server_negotiation.cc:299] Negotiation successful
0809 12:17:17.293117 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 12:17:17.297175 30031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:17.246987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:40421 (local address 127.0.0.1:50502)
0809 12:17:17.268647 (+ 21660us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:17.268670 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:17.268962 (+ 292us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:17.269592 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:17.269601 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:17.271199 (+ 1598us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:17.272513 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:17.272533 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.274402 (+ 1869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.274410 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:17.277511 (+ 3101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:17.277533 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.279618 (+ 2085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.279639 (+ 21us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:17.280208 (+ 569us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:17.281896 (+ 1688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:17.281937 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:17.283770 (+ 1833us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:17.288988 (+ 5218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:17.288992 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:17.288995 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:17.289606 (+ 611us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:17.290205 (+ 599us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:17.290209 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:17.290212 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:17.290441 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:17.291326 (+ 885us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:17.291330 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:17.292347 (+ 1017us) client_negotiation.cc:769] Sending connection context
0809 12:17:17.292544 (+ 197us) client_negotiation.cc:240] Negotiation successful
0809 12:17:17.293555 (+ 1011us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21465,"thread_start_us":10333,"threads_started":1}
Aug 09 12:17:17 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.183.193@KRBTEST.COM
I20260809 12:17:17.304488 30032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:17.246381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.0.0.1:43252)
0809 12:17:17.257918 (+ 11537us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:17.257940 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:17.258143 (+ 203us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:17.262593 (+ 4450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:17.262599 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:17.264672 (+ 2073us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:17.266643 (+ 1971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:17.266656 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.268044 (+ 1388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.268052 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:17.283029 (+ 14977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:17.283050 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.285335 (+ 2285us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.285346 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:17.285654 (+ 308us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:17.291970 (+ 6316us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:17.291998 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:17.295419 (+ 3421us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:17.300594 (+ 5175us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:17.300597 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:17.300600 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:17.301180 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:17.301793 (+ 613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:17.301798 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:17.301801 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:17.302027 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:17.303005 (+ 978us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:17.303011 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:17.303312 (+ 301us) client_negotiation.cc:769] Sending connection context
0809 12:17:17.303499 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 12:17:17.303916 (+ 417us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11351,"thread_start_us":13060,"threads_started":1}
I20260809 12:17:17.304513 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:17.242160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43252 (local address 127.19.183.254:34413)
0809 12:17:17.242346 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:17.242352 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:17.262042 (+ 19690us) server_negotiation.cc:407] Connection header received
0809 12:17:17.262137 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:17.262145 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:17.262254 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:17.262380 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:17.266925 (+ 4545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.267788 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.283272 (+ 15484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.284396 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.284555 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:17.295718 (+ 11163us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:17.295724 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:17.295728 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:17.295774 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:17.300433 (+ 4659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:17.301350 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:17.301374 (+ 24us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:17.301378 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:17.301643 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:17.302180 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:17.302185 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:17.302188 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:17.302820 (+ 632us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:17.303078 (+ 258us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:17.303742 (+ 664us) server_negotiation.cc:299] Negotiation successful
0809 12:17:17.304036 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 12:17:17.312099 30033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:17.273492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32950 (local address 127.19.183.193:37655)
0809 12:17:17.284960 (+ 11468us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:17.284973 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:17.285006 (+ 33us) server_negotiation.cc:407] Connection header received
0809 12:17:17.285080 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:17.285093 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:17.285512 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:17.288358 (+ 2846us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:17.288724 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.290912 (+ 2188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.293613 (+ 2701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.294933 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.295860 (+ 927us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:17.302431 (+ 6571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:17.302443 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:17.302458 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:17.302489 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:17.308177 (+ 5688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:17.309235 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:17.309243 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:17.309246 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:17.309489 (+ 243us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:17.310123 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:17.310128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:17.310131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:17.310523 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:17.310887 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:17.311267 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 12:17:17.311559 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11149,"thread_start_us":12208,"threads_started":1}
I20260809 12:17:17.314412 29186 generic_service.cc:196] Checking for leaks (request via RPC)
W20260809 12:17:17.320758 29918 tablet.cc:2369] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 12:17:17 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277834, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.254@KRBTEST.COM for oryx/127.19.183.252@KRBTEST.COM
I20260809 12:17:17.333808 29992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:17.311560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:41811 (local address 127.0.0.1:54208)
0809 12:17:17.312221 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:17.312241 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:17.312397 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:17.312878 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:17.312885 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:17.313335 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:17.313527 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:17.313536 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.315430 (+ 1894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.315436 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:17.317698 (+ 2262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:17.317719 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.319252 (+ 1533us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.319259 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:17.319648 (+ 389us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:17.320494 (+ 846us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:17.320523 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:17.324483 (+ 3960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:17.330241 (+ 5758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:17.330247 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:17.330251 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:17.331091 (+ 840us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:17.331623 (+ 532us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:17.331628 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:17.331632 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:17.331932 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:17.332716 (+ 784us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:17.332722 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:17.332913 (+ 191us) client_negotiation.cc:769] Sending connection context
0809 12:17:17.333095 (+ 182us) client_negotiation.cc:240] Negotiation successful
0809 12:17:17.333404 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506}
I20260809 12:17:17.334517 29994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:17.312079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54208 (local address 127.19.183.252:41811)
0809 12:17:17.312251 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:17.312256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:17.312280 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:17:17.312668 (+ 388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:17.312672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:17.312738 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:17.312847 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:17.313664 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.315264 (+ 1600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.317918 (+ 2654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.318984 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.319109 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:17.324713 (+ 5604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:17.324719 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:17.324725 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:17.324772 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:17.330089 (+ 5317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:17.331236 (+ 1147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:17.331241 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:17.331245 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:17.331464 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:17.332100 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:17.332105 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:17.332109 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:17.332565 (+ 456us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:17.332707 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:17.333267 (+ 560us) server_negotiation.cc:299] Negotiation successful
0809 12:17:17.333533 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:17:17.336154 28920 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.19.183.252:41811: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f"
member_type: NON_VOTER
last_known_addr {
host: "127.19.183.252"
port: 41811
}
attrs {
promote: true
}
}
dest_uuid: "7852578af3644424bdcd20194612fdac"
cas_config_opid_index: 5
I20260809 12:17:17.337455 28920 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: NON_VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: true } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: NON_VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: true } }
I20260809 12:17:17.339494 29987 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 5f59b93ddf0e4cc8be0df79193d05d8f (127.19.183.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: NON_VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: true } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: NON_VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: true } } }
I20260809 12:17:17.344399 29946 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:17.350194 29946 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 1 FOLLOWER]: Refusing update from remote peer 7852578af3644424bdcd20194612fdac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 12:17:17.366601 29831 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [term 1 LEARNER]: Refusing update from remote peer 7852578af3644424bdcd20194612fdac: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 12:17:17.376677 30038 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: NON_VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 12:17:17.381294 30037 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [LEADER]: Connected to new peer: Peer: permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: NON_VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.001s
I20260809 12:17:17.388801 29987 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.19.183.252:41811
I20260809 12:17:17.389451 29752 master.cc:537] Master@127.19.183.252:41811 shutting down...
I20260809 12:17:17.390934 30039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7852578af3644424bdcd20194612fdac" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: NON_VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: true } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: NON_VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: true } } }
I20260809 12:17:17.391510 30039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:17.396649 29831 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 5f59b93ddf0e4cc8be0df79193d05d8f (127.19.183.252) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: NON_VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: true } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: NON_VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: true } } }
I20260809 12:17:17.404273 29874 sys_catalog.cc:455] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7852578af3644424bdcd20194612fdac" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: NON_VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: true } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: NON_VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: true } } }
I20260809 12:17:17.404685 29874 sys_catalog.cc:458] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [sys.catalog]: This master's current role is: LEARNER
I20260809 12:17:17.405436 30039 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac: attempting to promote NON_VOTER b399be0c91ba480a873d964b624c404e to VOTER
I20260809 12:17:17.406889 30039 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: NON_VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: true } }
I20260809 12:17:17.413069 29946 raft_consensus.cc:858] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 1 NON_PARTICIPANT]: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: NON_VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: NON_VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: true } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: NON_VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: true } }
W20260809 12:17:17.413285 29946 raft_consensus.cc:1565] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 1 NON_PARTICIPANT]: Could not prepare op 'term: 1 index: 6' and following 0 ops. Status for this op: Illegal state: RaftConfig change currently pending. Only one is allowed at a time.
Committed config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } }.
Pending config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: NON_VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: true } }:
New config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: NON_VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: true } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: NON_VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: true } }
I20260809 12:17:17.415660 29831 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [term 1 LEARNER]: Refusing update from remote peer 7852578af3644424bdcd20194612fdac: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 12:17:17.416301 30036 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [LEADER]: Connected to new peer: Peer: permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 12:17:17.418915 30039 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b399be0c91ba480a873d964b624c404e (127.19.183.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: NON_VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: true } } }
I20260809 12:17:17.419589 29830 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b399be0c91ba480a873d964b624c404e (127.19.183.253) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: NON_VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: true } } }
I20260809 12:17:17.423189 30037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7852578af3644424bdcd20194612fdac" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: NON_VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: true } } }
I20260809 12:17:17.423501 30037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:17.423916 29874 sys_catalog.cc:455] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7852578af3644424bdcd20194612fdac" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: NON_VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: true } } }
I20260809 12:17:17.424170 29874 sys_catalog.cc:458] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:17:17.423969 30037 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7852578af3644424bdcd20194612fdac" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: NON_VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: true } } }
I20260809 12:17:17.424448 30037 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [sys.catalog]: This master's current role is: LEADER
W20260809 12:17:17.453611 28898 consensus_peers.cc:487] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac -> Peer 5f59b93ddf0e4cc8be0df79193d05d8f (127.19.183.252:41811): Couldn't send request to peer 5f59b93ddf0e4cc8be0df79193d05d8f. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 12:17:17.460985 29946 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 1 NON_PARTICIPANT]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER b399be0c91ba480a873d964b624c404e (127.19.183.253) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: NON_VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: true } } }
I20260809 12:17:17.466982 30000 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7852578af3644424bdcd20194612fdac. Latest consensus state: current_term: 1 leader_uuid: "7852578af3644424bdcd20194612fdac" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: NON_VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: true } } }
I20260809 12:17:17.466378 30053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7852578af3644424bdcd20194612fdac" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: NON_VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: true } } }
I20260809 12:17:17.468257 30000 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 12:17:17.466988 30050 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7852578af3644424bdcd20194612fdac" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: NON_VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: true } } }
I20260809 12:17:17.468480 30053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 12:17:17.468612 30050 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [sys.catalog]: This master's current role is: NON_PARTICIPANT
I20260809 12:17:17.468708 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 29004
I20260809 12:17:17.476999 29752 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 1 NON_PARTICIPANT]: Raft consensus shutting down.
I20260809 12:17:17.477502 29752 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 1 NON_PARTICIPANT]: Raft consensus is shut down!
I20260809 12:17:17.477684 29752 tablet_replica.cc:330] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f: stopping tablet replica
I20260809 12:17:17.484891 29752 master.cc:552] Master@127.19.183.252:41811 shutdown complete.
I20260809 12:17:17.485002 29752 master_runner.cc:305] Clearing existing system tablet
I20260809 12:17:17.485419 29752 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:17.485546 29752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:17.487962 29752 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 12:17:17.489657 30057 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:17.490005 29752 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:17.490302 29752 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260809 12:17:17.490408 29752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/wal
uuid: "5f59b93ddf0e4cc8be0df79193d05d8f"
format_stamp: "Formatted at 2026-08-09 12:17:17 on dist-test-slave-39l8"
server_key: "db5d4e5326f139845de32a118c231f87"
server_key_iv: "c61fd8a242ec7d1b1996db49aa368d6c"
server_key_version: "encryptionkey@0"
I20260809 12:17:17.493925 29752 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:17:17.498756 29752 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 12:17:17.498943 29752 log.cc:1196] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 12:17:17.499425 29752 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f: Deleting consensus metadata
I20260809 12:17:17.499678 29752 master_runner.cc:315] Copying system tablet from 127.19.183.254:34413
I20260809 12:17:17.502319 29752 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f: tablet copy: Beginning tablet copy session from remote peer at address 127.19.183.254:34413
I20260809 12:17:17.503893 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:37655
--local_ip_for_outbound_sockets=127.19.183.193
--tserver_master_addrs=127.19.183.254:34413
--webserver_port=38489
--webserver_interface=127.19.183.193
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.193.keytab
--principal=oryx/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:42821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.183.254:34413,127.19.183.253:40421
--tserver_master_addrs=127.19.183.254:34413,127.19.183.253:40421,127.19.183.252:41811 with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:17:17.522411 29997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:17.503121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43254 (local address 127.19.183.254:34413)
0809 12:17:17.503349 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:17.503354 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:17.510228 (+ 6874us) server_negotiation.cc:407] Connection header received
0809 12:17:17.510483 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:17.510487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:17.510558 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:17.510667 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:17.511490 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.512259 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.513503 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.513982 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.514152 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:17.516562 (+ 2410us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:17.516567 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:17.516570 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:17.516602 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:17.519662 (+ 3060us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:17.520327 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:17.520332 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:17.520336 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:17.520476 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:17.520873 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:17.520878 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:17.520882 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:17.521233 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:17.521395 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:17.521824 (+ 429us) server_negotiation.cc:299] Negotiation successful
0809 12:17:17.522042 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114}
I20260809 12:17:17.522495 30064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:17.503024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.0.0.1:43254)
0809 12:17:17.510098 (+ 7074us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:17.510130 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:17.510307 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:17.510717 (+ 410us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:17.510723 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:17.511140 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:17.511333 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:17.511342 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.512400 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.512406 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:17.513351 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:17.513365 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.514167 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.514173 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:17.514416 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:17.515210 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:17.515238 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:17.516370 (+ 1132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:17.519797 (+ 3427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:17.519804 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:17.519809 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:17.520197 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:17.520600 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:17.520609 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:17.520612 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:17.520754 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:17.521358 (+ 604us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:17.521363 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:17.521565 (+ 202us) client_negotiation.cc:769] Sending connection context
0809 12:17:17.521699 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 12:17:17.522071 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6872,"thread_start_us":6723,"threads_started":1}
I20260809 12:17:17.523413 28945 tablet_copy_service.cc:140] P 7852578af3644424bdcd20194612fdac: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 5f59b93ddf0e4cc8be0df79193d05d8f ({username='oryx', principal='oryx/127.19.183.252@KRBTEST.COM'} at 127.0.0.1:43254)
I20260809 12:17:17.523612 28945 tablet_copy_service.cc:161] P 7852578af3644424bdcd20194612fdac: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 5f59b93ddf0e4cc8be0df79193d05d8f at {username='oryx', principal='oryx/127.19.183.252@KRBTEST.COM'} at 127.0.0.1:43254: session id = 5f59b93ddf0e4cc8be0df79193d05d8f-00000000000000000000000000000000
I20260809 12:17:17.527742 28945 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 12:17:17.529899 29752 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 12:17:17.538046 29752 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f: tablet copy: Starting download of 0 data blocks...
I20260809 12:17:17.538440 29752 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f: tablet copy: Starting download of 1 WAL segments...
I20260809 12:17:17.541078 29752 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 12:17:17.545517 28945 tablet_copy_service.cc:342] P 7852578af3644424bdcd20194612fdac: Request end of tablet copy session 5f59b93ddf0e4cc8be0df79193d05d8f-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.19.183.252@KRBTEST.COM'} at 127.0.0.1:43254
I20260809 12:17:17.545650 28945 tablet_copy_service.cc:433] P 7852578af3644424bdcd20194612fdac: ending tablet copy session 5f59b93ddf0e4cc8be0df79193d05d8f-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 5f59b93ddf0e4cc8be0df79193d05d8f
I20260809 12:17:17.557494 29752 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:17.560184 30068 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:17.570573 30070 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:17.575104 29752 server_base.cc:1034] running on GCE node
W20260809 12:17:17.575855 30072 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:17.576367 29752 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:17.577028 29752 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:17.578233 29752 hybrid_clock.cc:648] HybridClock initialized: now 1786277837578193 us; error 31 us; skew 500 ppm
I20260809 12:17:17.578496 29752 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:17 dist-test-slave-39l8 krb5kdc[28862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277837, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:17.580775 29752 init.cc:377] Logged in from keytab as oryx/127.19.183.252@KRBTEST.COM (short username oryx)
I20260809 12:17:17.583712 29752 webserver.cc:460] Webserver started at http://127.19.183.252:33929/ using document root <none> and password file <none>
I20260809 12:17:17.584088 29752 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:17.584190 29752 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:17.587350 29752 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 12:17:17.589315 30081 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:17.589865 29752 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 12:17:17.589993 29752 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/wal
uuid: "5f59b93ddf0e4cc8be0df79193d05d8f"
format_stamp: "Formatted at 2026-08-09 12:17:17 on dist-test-slave-39l8"
server_key: "db5d4e5326f139845de32a118c231f87"
server_key_iv: "c61fd8a242ec7d1b1996db49aa368d6c"
server_key_version: "encryptionkey@0"
I20260809 12:17:17.590277 29752 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:17.603544 29752 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:17.607573 29752 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:17.632391 29752 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.252:41811
I20260809 12:17:17.632563 30147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.252:41811 every 8 connection(s)
I20260809 12:17:17.633461 29752 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/master-2/data/info.pb
I20260809 12:17:17.638445 30148 sys_catalog.cc:263] Verifying existing consensus state
I20260809 12:17:17.642257 30148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f: Bootstrap starting.
I20260809 12:17:17.658478 30148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:17:17.659161 30148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f: Bootstrap complete.
I20260809 12:17:17.660265 30148 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: NON_VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: true } }
I20260809 12:17:17.660693 30148 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 5f59b93ddf0e4cc8be0df79193d05d8f, State: Initialized, Role: LEARNER
I20260809 12:17:17.660955 30148 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: NON_VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: true } }
I20260809 12:17:17.661830 30151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: NON_VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: true } } }
I20260809 12:17:17.662108 30148 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:17:17.662269 30151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [sys.catalog]: This master's current role is: LEARNER
I20260809 12:17:17.674901 30175 catalog_manager.cc:1259] Loaded cluster ID: e474dd64bd7a4984a89c538000646418
I20260809 12:17:17.675024 30175 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f: loading cluster ID for follower catalog manager: success
I20260809 12:17:17.678475 30175 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f: acquiring CA information for follower catalog manager: success
I20260809 12:17:17.680958 30175 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 12:17:17.947340 30065 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:17.947757 30065 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:17.947939 30065 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260809 12:17:17.952275 30179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:17.945296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:41811 (local address 127.0.0.1:54210)
0809 12:17:17.945782 (+ 486us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:17.945802 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:17.945983 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:17.946410 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:17.946413 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:17.946436 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:17:17.946599 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:17.946606 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.948442 (+ 1836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.948447 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:17.949605 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:17.949617 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.951702 (+ 2085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.951706 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:17.951820 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:17.951843 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 12:17:17.951905 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 12:17:17.952004 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":218,"threads_started":1}
I20260809 12:17:17.952826 30182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:17.945426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54210 (local address 127.19.183.252:41811)
0809 12:17:17.946039 (+ 613us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:17.946045 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:17.946068 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:17:17.946144 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:17.946149 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:17.946249 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:17.946389 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:17:17.946744 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.948287 (+ 1543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.949764 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:17.951524 (+ 1760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:17.951771 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:17.951954 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:17.952022 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 12:17:17.952208 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":313,"threads_started":1}
I20260809 12:17:17.953927 30115 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260809 12:17:17.954402 30151 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7852578af3644424bdcd20194612fdac. Latest consensus state: current_term: 1 leader_uuid: "7852578af3644424bdcd20194612fdac" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: NON_VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: true } } }
I20260809 12:17:17.954839 30151 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [sys.catalog]: This master's current role is: LEARNER
W20260809 12:17:17.958666 30065 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:17.958894 30065 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:17.959097 30065 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:17:17.969921 30065 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:42821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.193.keytab
--principal=oryx/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:37655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=38489
--tserver_master_addrs=127.19.183.254:34413,127.19.183.253:40421,127.19.183.252:41811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:17.971323 30065 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:17.972782 30065 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:17.988231 30185 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:17.988183 30186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:17.988657 30189 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:17.989387 30065 server_base.cc:1034] running on GCE node
I20260809 12:17:17.990145 30065 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:17.991379 30065 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:17.992684 30065 hybrid_clock.cc:648] HybridClock initialized: now 1786277837992584 us; error 53 us; skew 500 ppm
I20260809 12:17:17.992995 30065 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:17 dist-test-slave-39l8 krb5kdc[28862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277837, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:17.997267 30065 init.cc:377] Logged in from keytab as oryx/127.19.183.193@KRBTEST.COM (short username oryx)
I20260809 12:17:17.999015 30065 webserver.cc:460] Webserver started at http://127.19.183.193:38489/ using document root <none> and password file <none>
I20260809 12:17:17.999768 30065 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:17.999933 30065 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:18.005925 30065 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.006s sys 0.000s
I20260809 12:17:18.008980 30201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:18.010399 30065 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:17:18.010596 30065 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "10c712c9a0d94e829c9c5c5763aa3277"
format_stamp: "Formatted at 2026-08-09 12:17:14 on dist-test-slave-39l8"
server_key: "45f7528a1523223872b5d847a2c858b3"
server_key_iv: "c67dac44488dc2d500708d4301e4e91d"
server_key_version: "encryptionkey@0"
I20260809 12:17:18.011933 30065 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:18.028723 30065 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:18.033892 30065 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:18.034256 30065 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:18.035382 30065 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:18.037384 30065 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:18.037528 30065 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:18.037683 30065 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:18.037817 30065 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:18.099261 30065 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:37655
I20260809 12:17:18.099326 30372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:37655 every 8 connection(s)
I20260809 12:17:18.102005 30065 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:17:18.112929 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 30065
I20260809 12:17:18.113363 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 29200
Aug 09 12:17:18 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277837, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.252@KRBTEST.COM
Aug 09 12:17:18 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.183.194@KRBTEST.COM
Aug 09 12:17:18 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277837, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM
I20260809 12:17:18.171207 30182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:18.108423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:57875 (local address 127.19.183.252:41811)
0809 12:17:18.108643 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:18.108649 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:18.113661 (+ 5012us) server_negotiation.cc:407] Connection header received
0809 12:17:18.119081 (+ 5420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:18.119088 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:18.119189 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:18.119324 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:18.132748 (+ 13424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.134409 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.141547 (+ 7138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.142463 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.142570 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:18.160701 (+ 18131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:18.160709 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:18.160714 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:18.160761 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:18.165912 (+ 5151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:18.167311 (+ 1399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:18.167316 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:18.167320 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:18.167515 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:18.168278 (+ 763us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:18.168285 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:18.168288 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:18.168778 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:18.168960 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:18.170392 (+ 1432us) server_negotiation.cc:299] Negotiation successful
0809 12:17:18.170700 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 12:17:18.174077 30379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:18.115548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35892 (local address 127.19.183.194:42409)
0809 12:17:18.116899 (+ 1351us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:18.116909 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:18.125985 (+ 9076us) server_negotiation.cc:407] Connection header received
0809 12:17:18.126063 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:18.126071 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:18.126366 (+ 295us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:18.126616 (+ 250us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:18.135271 (+ 8655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.137642 (+ 2371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.142230 (+ 4588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.143436 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.143599 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:18.161330 (+ 17731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:18.161352 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:18.161371 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:18.161417 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:18.167959 (+ 6542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:18.169452 (+ 1493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:18.169461 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:18.169464 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:18.169759 (+ 295us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:18.170450 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:18.170456 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:18.170459 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:18.171078 (+ 619us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:18.171553 (+ 475us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:18.172960 (+ 1407us) server_negotiation.cc:299] Negotiation successful
0809 12:17:18.173338 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1084,"thread_start_us":3402,"threads_started":1}
Aug 09 12:17:18 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277837, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.253@KRBTEST.COM
I20260809 12:17:18.174916 30376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:18.107776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:41811 (local address 127.19.183.193:57875)
0809 12:17:18.113376 (+ 5600us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:18.113480 (+ 104us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:18.118598 (+ 5118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:18.126279 (+ 7681us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:18.126291 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:18.127574 (+ 1283us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:18.128977 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:18.129016 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.135333 (+ 6317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.135339 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:18.138475 (+ 3136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:18.138500 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.147092 (+ 8592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.147117 (+ 25us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:18.148667 (+ 1550us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:18.153583 (+ 4916us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:18.153627 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:18.160290 (+ 6663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:18.166128 (+ 5838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:18.166138 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:18.166191 (+ 53us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:18.167140 (+ 949us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:18.167755 (+ 615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:18.167759 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:18.167762 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:18.168138 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:18.169127 (+ 989us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:18.169138 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:18.169688 (+ 550us) client_negotiation.cc:769] Sending connection context
0809 12:17:18.171827 (+ 2139us) client_negotiation.cc:240] Negotiation successful
0809 12:17:18.172662 (+ 835us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5089,"thread_start_us":6005,"threads_started":1}
I20260809 12:17:18.175944 29378 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:18.176419 30383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:18.119992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:48115 (local address 127.19.183.254:34413)
0809 12:17:18.141598 (+ 21606us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:18.141605 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:18.141624 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:17:18.141708 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:18.141714 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:18.141787 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:18.141952 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:18.145111 (+ 3159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.147708 (+ 2597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.153505 (+ 5797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.154790 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.154950 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:18.164426 (+ 9476us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:18.164433 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:18.164439 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:18.164483 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:18.169635 (+ 5152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:18.171093 (+ 1458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:18.171101 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:18.171105 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:18.171318 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:18.171810 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:18.171814 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:18.171817 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:18.172303 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:18.172442 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:18.175552 (+ 3110us) server_negotiation.cc:299] Negotiation successful
0809 12:17:18.175851 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21485,"thread_start_us":21305,"threads_started":1}
I20260809 12:17:18.177067 30382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:18.119836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.19.183.193:48115)
0809 12:17:18.139429 (+ 19593us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:18.139476 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:18.139686 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:18.142852 (+ 3166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:18.142857 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:18.144114 (+ 1257us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:18.144905 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:18.144922 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.148304 (+ 3382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.148313 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:18.153238 (+ 4925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:18.153266 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.156200 (+ 2934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.156210 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:18.156528 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:18.157661 (+ 1133us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:18.157694 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:18.164108 (+ 6414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:18.169866 (+ 5758us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:18.169870 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:18.169873 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:18.170895 (+ 1022us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:18.171473 (+ 578us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:18.171477 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:18.171480 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:18.171652 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:18.172523 (+ 871us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:18.172528 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:18.172935 (+ 407us) client_negotiation.cc:769] Sending connection context
0809 12:17:18.173096 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 12:17:18.173482 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19385,"thread_start_us":14848,"threads_started":1}
I20260809 12:17:18.177161 30373 heartbeater.cc:347] Connected to a master server at 127.19.183.252:41811
I20260809 12:17:18.177615 30373 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:18.179003 30373 heartbeater.cc:511] Master 127.19.183.252:41811 requested a full tablet report, sending...
I20260809 12:17:18.180296 30374 heartbeater.cc:347] Connected to a master server at 127.19.183.254:34413
I20260809 12:17:18.180544 30374 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:18.181767 30374 heartbeater.cc:511] Master 127.19.183.254:34413 requested a full tablet report, sending...
I20260809 12:17:18.181778 30101 ts_manager.cc:194] Registered new tserver with Master: 10c712c9a0d94e829c9c5c5763aa3277 (127.19.183.193:37655)
I20260809 12:17:18.185134 28920 ts_manager.cc:194] Re-registered known tserver with Master: 10c712c9a0d94e829c9c5c5763aa3277 (127.19.183.193:37655)
I20260809 12:17:18.186143 30388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:18.150875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:40421 (local address 127.19.183.193:42693)
0809 12:17:18.160128 (+ 9253us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:18.160144 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:18.160361 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:18.161146 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:18.161151 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:18.162442 (+ 1291us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:18.163403 (+ 961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:18.163434 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.165633 (+ 2199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.165642 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:18.168459 (+ 2817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:18.168476 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.170442 (+ 1966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.170450 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:18.170757 (+ 307us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:18.171773 (+ 1016us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:18.171797 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:18.175310 (+ 3513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:18.182646 (+ 7336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:18.182652 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:18.182656 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:18.183158 (+ 502us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:18.183591 (+ 433us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:18.183597 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:18.183601 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:18.183790 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:18.184634 (+ 844us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:18.184639 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:18.184904 (+ 265us) client_negotiation.cc:769] Sending connection context
0809 12:17:18.185094 (+ 190us) client_negotiation.cc:240] Negotiation successful
0809 12:17:18.185423 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9061,"thread_start_us":8256,"threads_started":1}
I20260809 12:17:18.186618 28920 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:48115
I20260809 12:17:18.187704 30387 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:18.151051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:42693 (local address 127.19.183.253:40421)
0809 12:17:18.159488 (+ 8437us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:18.159494 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:18.160264 (+ 770us) server_negotiation.cc:407] Connection header received
0809 12:17:18.160493 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:18.160499 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:18.160622 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:18.160804 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:18.163637 (+ 2833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.165352 (+ 1715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.168731 (+ 3379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.169801 (+ 1070us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.169895 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:18.175554 (+ 5659us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:18.175562 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:18.175567 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:18.175611 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:18.182512 (+ 6901us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:18.183286 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:18.183295 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:18.183298 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:18.183477 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:18.183915 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:18.183919 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:18.183921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:18.184461 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:18.186548 (+ 2087us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:18.186703 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 12:17:18.187010 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8282,"thread_start_us":7606,"threads_started":1}
I20260809 12:17:18.188941 30381 heartbeater.cc:347] Connected to a master server at 127.19.183.253:40421
I20260809 12:17:18.189153 30381 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:18.189764 30381 heartbeater.cc:511] Master 127.19.183.253:40421 requested a full tablet report, sending...
I20260809 12:17:18.191975 29819 ts_manager.cc:194] Registered new tserver with Master: 10c712c9a0d94e829c9c5c5763aa3277 (127.19.183.193:37655)
I20260809 12:17:18.272794 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 29200
I20260809 12:17:18.298658 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:42409
--local_ip_for_outbound_sockets=127.19.183.194
--tserver_master_addrs=127.19.183.254:34413
--webserver_port=35481
--webserver_interface=127.19.183.194
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.194.keytab
--principal=oryx/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:42821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.183.254:34413,127.19.183.253:40421
--tserver_master_addrs=127.19.183.254:34413,127.19.183.253:40421,127.19.183.252:41811 with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:17:18.506309 30039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7852578af3644424bdcd20194612fdac" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: NON_VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: true } } }
I20260809 12:17:18.506664 30039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:18.508389 30398 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac: attempting to promote NON_VOTER 5f59b93ddf0e4cc8be0df79193d05d8f to VOTER
I20260809 12:17:18.509035 30398 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 7, Committed index: 7, Last appended: 1.7, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: false } }
I20260809 12:17:18.513324 30115 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 1 LEARNER]: Refusing update from remote peer 7852578af3644424bdcd20194612fdac: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 12:17:18.513638 29830 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [term 1 FOLLOWER]: Refusing update from remote peer 7852578af3644424bdcd20194612fdac: Log matching property violated. Preceding OpId in replica: term: 1 index: 7. Preceding OpId from leader: term: 1 index: 8. (index mismatch)
I20260809 12:17:18.514061 30398 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 12:17:18.514400 30039 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [LEADER]: Connected to new peer: Peer: permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 8, Last known committed idx: 7, Time since last communication: 0.000s
I20260809 12:17:18.517613 30393 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [term 1 LEADER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 5f59b93ddf0e4cc8be0df79193d05d8f (127.19.183.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: false } } }
I20260809 12:17:18.518419 30115 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 5f59b93ddf0e4cc8be0df79193d05d8f (127.19.183.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: false } } }
I20260809 12:17:18.520792 30393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "7852578af3644424bdcd20194612fdac" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: false } } }
I20260809 12:17:18.521031 30393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:18.521366 30039 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7852578af3644424bdcd20194612fdac" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: false } } }
I20260809 12:17:18.521633 30039 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:18.521642 30393 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "7852578af3644424bdcd20194612fdac" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: false } } }
I20260809 12:17:18.521872 30393 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:18.522367 29830 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [term 1 FOLLOWER]: Committing config change with OpId 1.8: config changed from index 7 to 8, 5f59b93ddf0e4cc8be0df79193d05d8f (127.19.183.252) changed from NON_VOTER to VOTER. New config: { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: false } } }
I20260809 12:17:18.523157 30408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7852578af3644424bdcd20194612fdac" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: false } } }
I20260809 12:17:18.523466 30408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:17:18.526079 30409 sys_catalog.cc:455] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "7852578af3644424bdcd20194612fdac" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: false } } }
I20260809 12:17:18.526521 30409 sys_catalog.cc:458] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [sys.catalog]: This master's current role is: FOLLOWER
W20260809 12:17:18.645884 30392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:18.646144 30392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:18.646242 30392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:18.653595 30392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:18.653689 30392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:18.653800 30392 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:17:18.663628 30392 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:42821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.194.keytab
--principal=oryx/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:42409
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=35481
--tserver_master_addrs=127.19.183.254:34413,127.19.183.253:40421,127.19.183.252:41811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:18.664789 30392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:18.666085 30392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:18.679198 30421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:18.679164 30425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:18.679231 30422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:18.679970 30392 server_base.cc:1034] running on GCE node
I20260809 12:17:18.680593 30392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:18.681587 30392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:18.682806 30392 hybrid_clock.cc:648] HybridClock initialized: now 1786277838682736 us; error 55 us; skew 500 ppm
I20260809 12:17:18.683125 30392 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:18 dist-test-slave-39l8 krb5kdc[28862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277838, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:18.686280 30392 init.cc:377] Logged in from keytab as oryx/127.19.183.194@KRBTEST.COM (short username oryx)
I20260809 12:17:18.687758 30392 webserver.cc:460] Webserver started at http://127.19.183.194:35481/ using document root <none> and password file <none>
I20260809 12:17:18.688441 30392 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:18.688606 30392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:18.694402 30392 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.001s
I20260809 12:17:18.697052 30439 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:18.698335 30392 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 12:17:18.698510 30392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "2e535068581c4dbcaca1b42f72e5b911"
format_stamp: "Formatted at 2026-08-09 12:17:15 on dist-test-slave-39l8"
server_key: "692b3dcfe76f7ea41d1ffa7942b158bd"
server_key_iv: "2d2cc4dfd885d9f4a9ef9c6124b6a8b5"
server_key_version: "encryptionkey@0"
I20260809 12:17:18.699572 30392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:18.732048 30392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:18.735962 30392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:18.736279 30392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:18.737248 30392 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:18.738580 30392 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:18.738698 30392 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:18.738854 30392 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:18.738953 30392 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:18.776540 30392 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:42409
I20260809 12:17:18.776628 30607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:42409 every 8 connection(s)
I20260809 12:17:18.777993 30392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:17:18.783418 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 30392
I20260809 12:17:18.783753 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 29398
Aug 09 12:17:18 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277838, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.252@KRBTEST.COM
Aug 09 12:17:18 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277833, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.183.195@KRBTEST.COM
Aug 09 12:17:18 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277838, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.253@KRBTEST.COM
Aug 09 12:17:18 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277838, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM
I20260809 12:17:18.841775 30622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:18.782856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:48863 (local address 127.19.183.252:41811)
0809 12:17:18.807670 (+ 24814us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:18.807681 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:18.807707 (+ 26us) server_negotiation.cc:407] Connection header received
0809 12:17:18.807788 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:18.807796 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:18.807911 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:18.808138 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:18.813346 (+ 5208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.815023 (+ 1677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.818449 (+ 3426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.819574 (+ 1125us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.819704 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:18.826526 (+ 6822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:18.826533 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:18.826538 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:18.826577 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:18.835678 (+ 9101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:18.836931 (+ 1253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:18.836937 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:18.836941 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:18.837165 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:18.838666 (+ 1501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:18.838672 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:18.838676 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:18.839402 (+ 726us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:18.839575 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:18.840673 (+ 1098us) server_negotiation.cc:299] Negotiation successful
0809 12:17:18.840957 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24642,"thread_start_us":26370,"threads_started":1}
I20260809 12:17:18.843241 30621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:18.782073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:41811 (local address 127.19.183.194:48863)
0809 12:17:18.798560 (+ 16487us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:18.798637 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:18.802288 (+ 3651us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:18.809804 (+ 7516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:18.809819 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:18.810898 (+ 1079us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:18.813124 (+ 2226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:18.813172 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.815291 (+ 2119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.815300 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:18.818224 (+ 2924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:18.818247 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.820331 (+ 2084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.820342 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:18.820847 (+ 505us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:18.821915 (+ 1068us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:18.821957 (+ 42us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:18.826300 (+ 4343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:18.835834 (+ 9534us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:18.835845 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:18.835865 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:18.836759 (+ 894us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:18.837345 (+ 586us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:18.837364 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:18.837370 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:18.838379 (+ 1009us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:18.839578 (+ 1199us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:18.839587 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:18.840106 (+ 519us) client_negotiation.cc:769] Sending connection context
0809 12:17:18.840524 (+ 418us) client_negotiation.cc:240] Negotiation successful
0809 12:17:18.841112 (+ 588us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15112,"thread_start_us":26242,"threads_started":1}
I20260809 12:17:18.845546 30610 heartbeater.cc:347] Connected to a master server at 127.19.183.252:41811
I20260809 12:17:18.846148 30610 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:18.847882 30625 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:18.785408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48686 (local address 127.19.183.195:36023)
0809 12:17:18.812741 (+ 27333us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:18.812755 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:18.812792 (+ 37us) server_negotiation.cc:407] Connection header received
0809 12:17:18.812872 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:18.812884 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:18.813152 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:18.813395 (+ 243us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:18.816405 (+ 3010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.820433 (+ 4028us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.824020 (+ 3587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.824819 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.824919 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:18.836654 (+ 11735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:18.836666 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:18.836680 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:18.836712 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:18.842590 (+ 5878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:18.843581 (+ 991us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:18.843589 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:18.843592 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:18.843838 (+ 246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:18.844527 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:18.844532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:18.844535 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:18.845043 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:18.845478 (+ 435us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:18.846804 (+ 1326us) server_negotiation.cc:299] Negotiation successful
0809 12:17:18.847142 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27098,"thread_start_us":32674,"threads_started":1}
I20260809 12:17:18.847857 30610 heartbeater.cc:511] Master 127.19.183.252:41811 requested a full tablet report, sending...
I20260809 12:17:18.849659 29580 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:18.849740 30623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:18.782905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:40421 (local address 127.19.183.194:54915)
0809 12:17:18.812028 (+ 29123us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:18.812050 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:18.812187 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:18.821836 (+ 9649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:18.821841 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:18.823391 (+ 1550us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:18.825444 (+ 2053us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:18.825460 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.827420 (+ 1960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.827426 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:18.828927 (+ 1501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:18.828940 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.830117 (+ 1177us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.830126 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:18.830366 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:18.831720 (+ 1354us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:18.831780 (+ 60us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:18.835353 (+ 3573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:18.841302 (+ 5949us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:18.841307 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:18.841310 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:18.842268 (+ 958us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:18.842831 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:18.842836 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:18.842839 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:18.843061 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:18.844155 (+ 1094us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:18.844161 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:18.844442 (+ 281us) client_negotiation.cc:769] Sending connection context
0809 12:17:18.844614 (+ 172us) client_negotiation.cc:240] Negotiation successful
0809 12:17:18.849088 (+ 4474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28996,"mutex_wait_us":24,"thread_start_us":28866,"threads_started":1}
I20260809 12:17:18.849962 30618 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:18.781312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:54915 (local address 127.19.183.253:40421)
0809 12:17:18.819952 (+ 38640us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:18.819959 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:18.819977 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:17:18.820082 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:18.820087 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:18.820181 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:18.821818 (+ 1637us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:18.825708 (+ 3890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.827202 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.829078 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.829764 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.829906 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:18.835568 (+ 5662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:18.835573 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:18.835576 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:18.835615 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:18.841113 (+ 5498us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:18.842419 (+ 1306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:18.842424 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:18.842427 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:18.842654 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:18.843283 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:18.843289 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:18.843293 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:18.843835 (+ 542us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:18.844031 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:18.846062 (+ 2031us) server_negotiation.cc:299] Negotiation successful
0809 12:17:18.846753 (+ 691us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38511,"thread_start_us":39936,"threads_started":1}
I20260809 12:17:18.850663 30101 ts_manager.cc:194] Registered new tserver with Master: 2e535068581c4dbcaca1b42f72e5b911 (127.19.183.194:42409)
I20260809 12:17:18.851711 30614 heartbeater.cc:347] Connected to a master server at 127.19.183.253:40421
I20260809 12:17:18.851903 30614 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:18.852607 30614 heartbeater.cc:511] Master 127.19.183.253:40421 requested a full tablet report, sending...
I20260809 12:17:18.854166 29819 ts_manager.cc:194] Registered new tserver with Master: 2e535068581c4dbcaca1b42f72e5b911 (127.19.183.194:42409)
I20260809 12:17:18.858625 30620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:18.782074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.19.183.194:33083)
0809 12:17:18.811215 (+ 29141us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:18.811247 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:18.811470 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:18.823064 (+ 11594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:18.823070 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:18.824018 (+ 948us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:18.825593 (+ 1575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:18.825610 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.827820 (+ 2210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.827837 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:18.830567 (+ 2730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:18.830587 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.832587 (+ 2000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.832609 (+ 22us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:18.832971 (+ 362us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:18.836038 (+ 3067us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:18.836063 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:18.844601 (+ 8538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:18.854872 (+ 10271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:18.854881 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:18.854888 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:18.855811 (+ 923us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:18.856386 (+ 575us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:18.856391 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:18.856395 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:18.856694 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:18.857571 (+ 877us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:18.857577 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:18.857798 (+ 221us) client_negotiation.cc:769] Sending connection context
0809 12:17:18.857985 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 12:17:18.858233 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26548,"mutex_wait_us":62,"spinlock_wait_cycles":31488,"thread_start_us":14723,"threads_started":1}
I20260809 12:17:18.859431 30616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:18.781205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:33083 (local address 127.19.183.254:34413)
0809 12:17:18.822531 (+ 41326us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:18.822537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:18.822557 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:17:18.822658 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:18.822663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:18.822833 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:18.823044 (+ 211us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:18.825962 (+ 2918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.827308 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.830764 (+ 3456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:18.831343 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:18.831429 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:18.844867 (+ 13438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:18.844873 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:18.844879 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:18.844959 (+ 80us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:18.851087 (+ 6128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:18.856002 (+ 4915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:18.856007 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:18.856011 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:18.856250 (+ 239us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:18.856819 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:18.856825 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:18.856828 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:18.857374 (+ 546us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:18.857598 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:18.858230 (+ 632us) server_negotiation.cc:299] Negotiation successful
0809 12:17:18.858573 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41118,"thread_start_us":40960,"threads_started":1}
I20260809 12:17:18.860333 30613 heartbeater.cc:347] Connected to a master server at 127.19.183.254:34413
I20260809 12:17:18.860548 30613 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:18.861254 30613 heartbeater.cc:511] Master 127.19.183.254:34413 requested a full tablet report, sending...
I20260809 12:17:18.862598 28920 ts_manager.cc:194] Re-registered known tserver with Master: 2e535068581c4dbcaca1b42f72e5b911 (127.19.183.194:42409)
I20260809 12:17:18.864354 28920 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:33083
I20260809 12:17:18.949105 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 29398
I20260809 12:17:18.973907 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:36023
--local_ip_for_outbound_sockets=127.19.183.195
--tserver_master_addrs=127.19.183.254:34413
--webserver_port=42139
--webserver_interface=127.19.183.195
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.195.keytab
--principal=oryx/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:42821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.19.183.254:34413,127.19.183.253:40421
--tserver_master_addrs=127.19.183.254:34413,127.19.183.253:40421,127.19.183.252:41811 with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:17:19.190908 30374 heartbeater.cc:503] Master 127.19.183.254:34413 was elected leader, sending a full tablet report...
W20260809 12:17:19.318209 30627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:19.318540 30627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:19.318729 30627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:19.325865 30627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:19.325986 30627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:19.326367 30627 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:17:19.336097 30627 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:42821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.195.keytab
--principal=oryx/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:36023
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=42139
--tserver_master_addrs=127.19.183.254:34413,127.19.183.253:40421,127.19.183.252:41811
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:19.337400 30627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:19.338789 30627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:19.352504 30635 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:19.352573 30636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:19.352599 30639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:19.353790 30627 server_base.cc:1034] running on GCE node
I20260809 12:17:19.354349 30627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:19.355332 30627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:19.356549 30627 hybrid_clock.cc:648] HybridClock initialized: now 1786277839356509 us; error 50 us; skew 500 ppm
I20260809 12:17:19.356889 30627 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:19 dist-test-slave-39l8 krb5kdc[28862](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277839, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:19.360198 30627 init.cc:377] Logged in from keytab as oryx/127.19.183.195@KRBTEST.COM (short username oryx)
I20260809 12:17:19.361539 30627 webserver.cc:460] Webserver started at http://127.19.183.195:42139/ using document root <none> and password file <none>
I20260809 12:17:19.362234 30627 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:19.362368 30627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:19.367835 30627 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:17:19.370543 30652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:19.371836 30627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:17:19.372016 30627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "565f367d64c347598f718368daa08a26"
format_stamp: "Formatted at 2026-08-09 12:17:15 on dist-test-slave-39l8"
server_key: "f451d35b316d6da01e8c351518f0655b"
server_key_iv: "de04fe442763e20fe9b66974630c5b05"
server_key_version: "encryptionkey@0"
I20260809 12:17:19.372979 30627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:19.387799 30627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:19.392493 30627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:19.392787 30627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:19.393851 30627 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:19.395287 30627 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:19.395398 30627 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:19.395525 30627 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:19.395622 30627 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:19.429620 30627 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:36023
I20260809 12:17:19.429710 30812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:36023 every 8 connection(s)
I20260809 12:17:19.431634 30627 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:17:19.441462 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 30627
Aug 09 12:17:19 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277839, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.252@KRBTEST.COM
Aug 09 12:17:19 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277839, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM
Aug 09 12:17:19 dist-test-slave-39l8 krb5kdc[28862](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277839, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.253@KRBTEST.COM
I20260809 12:17:19.496757 30828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:19.435596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:40421 (local address 127.19.183.195:49391)
0809 12:17:19.460274 (+ 24678us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:19.460340 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:19.460576 (+ 236us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:19.461271 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:19.461278 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:19.462490 (+ 1212us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:19.463385 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:19.463400 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.465087 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.465095 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:19.468009 (+ 2914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:19.468025 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.469846 (+ 1821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.469856 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:19.470233 (+ 377us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:19.472398 (+ 2165us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:19.472428 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:19.481781 (+ 9353us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:19.488735 (+ 6954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:19.488740 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:19.488744 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:19.489680 (+ 936us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:19.490494 (+ 814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:19.490499 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:19.490503 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:19.492226 (+ 1723us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:19.493689 (+ 1463us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:19.493698 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:19.494176 (+ 478us) client_negotiation.cc:769] Sending connection context
0809 12:17:19.494581 (+ 405us) client_negotiation.cc:240] Negotiation successful
0809 12:17:19.495151 (+ 570us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24454,"mutex_wait_us":1456,"spinlock_wait_cycles":1628032,"thread_start_us":17978,"threads_started":1}
I20260809 12:17:19.498570 30825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:19.435596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.19.183.195:58317)
0809 12:17:19.448095 (+ 12499us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:19.448141 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:19.450625 (+ 2484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:19.458020 (+ 7395us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:19.458024 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:19.459166 (+ 1142us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:19.460543 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:19.460588 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.464491 (+ 3903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.464499 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:19.468021 (+ 3522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:19.468038 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.471492 (+ 3454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.471500 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:19.471813 (+ 313us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:19.475379 (+ 3566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:19.475401 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:19.480877 (+ 5476us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:19.486562 (+ 5685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:19.486572 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:19.486588 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:19.487388 (+ 800us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:19.490804 (+ 3416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:19.490811 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:19.490817 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:19.492046 (+ 1229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:19.497130 (+ 5084us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:19.497135 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:19.497337 (+ 202us) client_negotiation.cc:769] Sending connection context
0809 12:17:19.497484 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 12:17:19.497854 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12205,"mutex_wait_us":77,"spinlock_wait_cycles":17792,"thread_start_us":11963,"threads_started":1}
I20260809 12:17:19.498736 30827 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:19.435919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:58317 (local address 127.19.183.254:34413)
0809 12:17:19.456547 (+ 20628us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:19.456553 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:19.456571 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:17:19.456632 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:19.456638 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:19.456716 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:19.458007 (+ 1291us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:19.460773 (+ 2766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.464323 (+ 3550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.468235 (+ 3912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.469273 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.469385 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:19.481095 (+ 11710us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:19.481128 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:19.481133 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:19.481175 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:19.486379 (+ 5204us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:19.487520 (+ 1141us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:19.487527 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:19.487532 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:19.487789 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:19.493964 (+ 6175us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:19.493970 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:19.493975 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:19.494416 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:19.494597 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:19.497864 (+ 3267us) server_negotiation.cc:299] Negotiation successful
0809 12:17:19.498175 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20504,"thread_start_us":20393,"threads_started":1}
I20260809 12:17:19.499816 30819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:19.434902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:49391 (local address 127.19.183.253:40421)
0809 12:17:19.458948 (+ 24046us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:19.458954 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:19.460731 (+ 1777us) server_negotiation.cc:407] Connection header received
0809 12:17:19.460798 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:19.460803 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:19.460912 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:19.461062 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:19.463599 (+ 2537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.464924 (+ 1325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.468193 (+ 3269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.469172 (+ 979us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.469293 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:19.482059 (+ 12766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:19.482064 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:19.482078 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:19.482131 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:19.488530 (+ 6399us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:19.489919 (+ 1389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:19.489924 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:19.489927 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:19.490302 (+ 375us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:19.492440 (+ 2138us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:19.492447 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:19.492452 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:19.493419 (+ 967us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:19.493586 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:19.499006 (+ 5420us) server_negotiation.cc:299] Negotiation successful
0809 12:17:19.499279 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23929,"thread_start_us":23722,"threads_started":1}
I20260809 12:17:19.500492 30816 heartbeater.cc:347] Connected to a master server at 127.19.183.254:34413
I20260809 12:17:19.500952 30816 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:19.501559 30817 heartbeater.cc:347] Connected to a master server at 127.19.183.253:40421
I20260809 12:17:19.501778 30817 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:19.502640 30820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:19.434852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:57463 (local address 127.19.183.252:41811)
0809 12:17:19.447459 (+ 12607us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:19.447469 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:19.455460 (+ 7991us) server_negotiation.cc:407] Connection header received
0809 12:17:19.455834 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:19.455841 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:19.455965 (+ 124us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:19.456110 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:19.461318 (+ 5208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.463512 (+ 2194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.470440 (+ 6928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.471512 (+ 1072us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.471646 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:19.489284 (+ 17638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:19.489291 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:19.489296 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:19.489355 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:19.495093 (+ 5738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:19.496070 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:19.496075 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:19.496078 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:19.496262 (+ 184us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:19.497114 (+ 852us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:19.497128 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:19.497131 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:19.497566 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:19.497702 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:19.499660 (+ 1958us) server_negotiation.cc:299] Negotiation successful
0809 12:17:19.502093 (+ 2433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12412,"thread_start_us":15887,"threads_started":1}
I20260809 12:17:19.503511 30817 heartbeater.cc:511] Master 127.19.183.253:40421 requested a full tablet report, sending...
I20260809 12:17:19.505019 30816 heartbeater.cc:511] Master 127.19.183.254:34413 requested a full tablet report, sending...
I20260809 12:17:19.505834 29819 ts_manager.cc:194] Registered new tserver with Master: 565f367d64c347598f718368daa08a26 (127.19.183.195:36023)
I20260809 12:17:19.505887 30829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:19.436394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:41811 (local address 127.19.183.195:57463)
0809 12:17:19.455346 (+ 18952us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:19.455361 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:19.455523 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:19.456676 (+ 1153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:19.456687 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:19.459425 (+ 2738us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:19.461074 (+ 1649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:19.461095 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.463804 (+ 2709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.463812 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:19.470265 (+ 6453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:19.470279 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.472139 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.472150 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:19.472398 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:19.474916 (+ 2518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:19.474940 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:19.480808 (+ 5868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:19.495252 (+ 14444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:19.495256 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:19.495259 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:19.495931 (+ 672us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:19.496458 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:19.496463 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:19.496466 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:19.496713 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:19.497829 (+ 1116us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:19.497833 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:19.498026 (+ 193us) client_negotiation.cc:769] Sending connection context
0809 12:17:19.498255 (+ 229us) client_negotiation.cc:240] Negotiation successful
0809 12:17:19.498589 (+ 334us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18843,"mutex_wait_us":5,"thread_start_us":23413,"threads_started":1}
I20260809 12:17:19.506676 28920 ts_manager.cc:194] Re-registered known tserver with Master: 565f367d64c347598f718368daa08a26 (127.19.183.195:36023)
I20260809 12:17:19.507936 28920 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:58317
I20260809 12:17:19.509019 30814 heartbeater.cc:347] Connected to a master server at 127.19.183.252:41811
I20260809 12:17:19.509255 30814 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:19.510250 30814 heartbeater.cc:511] Master 127.19.183.252:41811 requested a full tablet report, sending...
I20260809 12:17:19.511969 30101 ts_manager.cc:194] Registered new tserver with Master: 565f367d64c347598f718368daa08a26 (127.19.183.195:36023)
I20260809 12:17:19.513485 30839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:19.469408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43268 (local address 127.19.183.254:34413)
0809 12:17:19.474470 (+ 5062us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:19.474476 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:19.477094 (+ 2618us) server_negotiation.cc:407] Connection header received
0809 12:17:19.477167 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:19.477174 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:19.477255 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:19.477371 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:19.482856 (+ 5485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.484471 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.488177 (+ 3706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.489192 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.489388 (+ 196us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:19.495080 (+ 5692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:19.495086 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:19.495091 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:19.495129 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:19.501226 (+ 6097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:19.503022 (+ 1796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:19.503029 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:19.503032 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:19.503406 (+ 374us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:19.508633 (+ 5227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:19.508643 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:19.508647 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:19.509061 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:19.509488 (+ 427us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:19.512672 (+ 3184us) server_negotiation.cc:299] Negotiation successful
0809 12:17:19.512968 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4864,"thread_start_us":4712,"threads_started":1}
I20260809 12:17:19.517411 30842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:19.471555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54214 (local address 127.19.183.252:41811)
0809 12:17:19.473726 (+ 2171us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:19.473732 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:19.484011 (+ 10279us) server_negotiation.cc:407] Connection header received
0809 12:17:19.484718 (+ 707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:19.484725 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:19.484819 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:19.484995 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:19.491354 (+ 6359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.493738 (+ 2384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.499924 (+ 6186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.500954 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.501057 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:19.507219 (+ 6162us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:19.507228 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:19.507236 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:19.507276 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:19.513225 (+ 5949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:19.514343 (+ 1118us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:19.514349 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:19.514354 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:19.514648 (+ 294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:19.515367 (+ 719us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:19.515373 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:19.515378 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:19.515831 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:19.516074 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:19.516626 (+ 552us) server_negotiation.cc:299] Negotiation successful
0809 12:17:19.516857 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2019,"thread_start_us":1786,"threads_started":1}
I20260809 12:17:19.523402 30838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:19.469154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50514 (local address 127.19.183.253:40421)
0809 12:17:19.475214 (+ 6060us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:19.475221 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:19.484588 (+ 9367us) server_negotiation.cc:407] Connection header received
0809 12:17:19.487365 (+ 2777us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:19.487372 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:19.487470 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:19.487579 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:19.504030 (+ 16451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.505715 (+ 1685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.508307 (+ 2592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.509494 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.509586 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:19.515987 (+ 6401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:19.515995 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:19.515999 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:19.516035 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:19.520014 (+ 3979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:19.520742 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:19.520748 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:19.520751 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:19.520963 (+ 212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:19.521501 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:19.521507 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:19.521510 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:19.521771 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:19.522031 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:19.522626 (+ 595us) server_negotiation.cc:299] Negotiation successful
0809 12:17:19.522912 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5829,"thread_start_us":5696,"threads_started":1}
I20260809 12:17:19.519639 28920 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43268:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:17:19.529392 28920 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:17:19.563655 30869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:19.550090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:36023 (local address 127.0.0.1:48698)
0809 12:17:19.551937 (+ 1847us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:19.551961 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:19.552319 (+ 358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:19.553457 (+ 1138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:19.553462 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:19.553508 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:17:19.554007 (+ 499us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:19.554024 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.556707 (+ 2683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.556715 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:19.560549 (+ 3834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:19.560563 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.562593 (+ 2030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.562601 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:19.562858 (+ 257us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:19.562887 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 12:17:19.562994 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 12:17:19.563159 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1679,"threads_started":1}
I20260809 12:17:19.563874 30865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:19.550362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48698 (local address 127.19.183.195:36023)
0809 12:17:19.551216 (+ 854us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:19.551229 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:19.552166 (+ 937us) server_negotiation.cc:407] Connection header received
0809 12:17:19.552557 (+ 391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:19.552569 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:19.552882 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:19.553079 (+ 197us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:17:19.554322 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.556459 (+ 2137us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.560733 (+ 4274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.562266 (+ 1533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.562483 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:19.562697 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:19.563107 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 12:17:19.563361 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":669,"thread_start_us":677,"threads_started":1}
I20260809 12:17:19.568954 30721 tablet_service.cc:1467] Processing CreateTablet for tablet 73ce8294c39c43e987119dab03ba66fc (DEFAULT_TABLE table=test-table [id=37c782b20afb4a9ebf92d8682e436dea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:17:19.570307 30721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73ce8294c39c43e987119dab03ba66fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:19.576750 30863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:19.551113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:42409 (local address 127.0.0.1:35896)
0809 12:17:19.563251 (+ 12138us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:19.563272 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:19.563560 (+ 288us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:19.569854 (+ 6294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:19.569859 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:19.569924 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:17:19.570857 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:19.570871 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.572449 (+ 1578us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.572453 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:19.574305 (+ 1852us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:19.574318 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.575869 (+ 1551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.575876 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:19.576095 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:19.576129 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 12:17:19.576231 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 12:17:19.576362 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11880,"mutex_wait_us":3,"thread_start_us":1836,"threads_started":1}
I20260809 12:17:19.577837 30874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:19.552267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35896 (local address 127.19.183.194:42409)
0809 12:17:19.567351 (+ 15084us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:19.567362 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:19.567397 (+ 35us) server_negotiation.cc:407] Connection header received
0809 12:17:19.567509 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:19.567516 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:19.567816 (+ 300us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:19.568131 (+ 315us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:17:19.571052 (+ 2921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.572280 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.574468 (+ 2188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.575641 (+ 1173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.575904 (+ 263us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:19.576113 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:19.576296 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 12:17:19.576550 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":14889,"thread_start_us":14785,"threads_started":1}
I20260809 12:17:19.579479 30872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:19.551698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32958 (local address 127.19.183.193:37655)
0809 12:17:19.569030 (+ 17332us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:19.569038 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:19.569063 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:17:19.569142 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:19.569147 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:19.569301 (+ 154us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:19.569493 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:17:19.570432 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.572358 (+ 1926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.576115 (+ 3757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.578069 (+ 1954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.578254 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:19.578409 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:19.579023 (+ 614us) server_negotiation.cc:299] Negotiation successful
0809 12:17:19.579187 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17092,"thread_start_us":16955,"threads_started":1}
I20260809 12:17:19.579818 30866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:19.550836 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:37655 (local address 127.0.0.1:32958)
0809 12:17:19.560674 (+ 9838us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:19.560696 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:19.561062 (+ 366us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:19.569485 (+ 8423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:19.569489 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:19.569523 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:17:19.570140 (+ 617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:19.570159 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.572577 (+ 2418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.572584 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:19.575854 (+ 3270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:19.575874 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.578568 (+ 2694us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.578575 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:19.578810 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:19.578852 (+ 42us) client_negotiation.cc:769] Sending connection context
0809 12:17:19.578954 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 12:17:19.579091 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9686,"mutex_wait_us":27,"thread_start_us":9461,"threads_started":1}
I20260809 12:17:19.581027 30511 tablet_service.cc:1467] Processing CreateTablet for tablet 73ce8294c39c43e987119dab03ba66fc (DEFAULT_TABLE table=test-table [id=37c782b20afb4a9ebf92d8682e436dea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:17:19.582028 30511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73ce8294c39c43e987119dab03ba66fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:19.585918 30875 tablet_bootstrap.cc:492] T 73ce8294c39c43e987119dab03ba66fc P 565f367d64c347598f718368daa08a26: Bootstrap starting.
I20260809 12:17:19.586463 30275 tablet_service.cc:1467] Processing CreateTablet for tablet 73ce8294c39c43e987119dab03ba66fc (DEFAULT_TABLE table=test-table [id=37c782b20afb4a9ebf92d8682e436dea]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:17:19.587775 30275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 73ce8294c39c43e987119dab03ba66fc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:19.589627 30875 tablet_bootstrap.cc:654] T 73ce8294c39c43e987119dab03ba66fc P 565f367d64c347598f718368daa08a26: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:19.592344 30875 log.cc:824] T 73ce8294c39c43e987119dab03ba66fc P 565f367d64c347598f718368daa08a26: Log is configured to *not* fsync() on all Append() calls
I20260809 12:17:19.596853 30875 tablet_bootstrap.cc:492] T 73ce8294c39c43e987119dab03ba66fc P 565f367d64c347598f718368daa08a26: No bootstrap required, opened a new log
I20260809 12:17:19.597270 30875 ts_tablet_manager.cc:1397] T 73ce8294c39c43e987119dab03ba66fc P 565f367d64c347598f718368daa08a26: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260809 12:17:19.600621 30877 tablet_bootstrap.cc:492] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911: Bootstrap starting.
I20260809 12:17:19.606580 30877 tablet_bootstrap.cc:654] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:19.607410 30878 tablet_bootstrap.cc:492] T 73ce8294c39c43e987119dab03ba66fc P 10c712c9a0d94e829c9c5c5763aa3277: Bootstrap starting.
I20260809 12:17:19.608547 30877 log.cc:824] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911: Log is configured to *not* fsync() on all Append() calls
I20260809 12:17:19.612475 30877 tablet_bootstrap.cc:492] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911: No bootstrap required, opened a new log
I20260809 12:17:19.612743 30878 tablet_bootstrap.cc:654] T 73ce8294c39c43e987119dab03ba66fc P 10c712c9a0d94e829c9c5c5763aa3277: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:19.612802 30877 ts_tablet_manager.cc:1397] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911: Time spent bootstrapping tablet: real 0.013s user 0.001s sys 0.009s
I20260809 12:17:19.613226 30875 raft_consensus.cc:348] T 73ce8294c39c43e987119dab03ba66fc P 565f367d64c347598f718368daa08a26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565f367d64c347598f718368daa08a26" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36023 } } peers { permanent_uuid: "10c712c9a0d94e829c9c5c5763aa3277" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37655 } } peers { permanent_uuid: "2e535068581c4dbcaca1b42f72e5b911" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42409 } }
I20260809 12:17:19.614009 30875 raft_consensus.cc:374] T 73ce8294c39c43e987119dab03ba66fc P 565f367d64c347598f718368daa08a26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:19.614228 30875 raft_consensus.cc:729] T 73ce8294c39c43e987119dab03ba66fc P 565f367d64c347598f718368daa08a26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 565f367d64c347598f718368daa08a26, State: Initialized, Role: FOLLOWER
I20260809 12:17:19.615228 30878 log.cc:824] T 73ce8294c39c43e987119dab03ba66fc P 10c712c9a0d94e829c9c5c5763aa3277: Log is configured to *not* fsync() on all Append() calls
I20260809 12:17:19.615202 30875 consensus_queue.cc:260] T 73ce8294c39c43e987119dab03ba66fc P 565f367d64c347598f718368daa08a26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565f367d64c347598f718368daa08a26" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36023 } } peers { permanent_uuid: "10c712c9a0d94e829c9c5c5763aa3277" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37655 } } peers { permanent_uuid: "2e535068581c4dbcaca1b42f72e5b911" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42409 } }
I20260809 12:17:19.618796 30875 ts_tablet_manager.cc:1428] T 73ce8294c39c43e987119dab03ba66fc P 565f367d64c347598f718368daa08a26: Time spent starting tablet: real 0.021s user 0.012s sys 0.011s
I20260809 12:17:19.619185 30816 heartbeater.cc:503] Master 127.19.183.254:34413 was elected leader, sending a full tablet report...
I20260809 12:17:19.619689 30878 tablet_bootstrap.cc:492] T 73ce8294c39c43e987119dab03ba66fc P 10c712c9a0d94e829c9c5c5763aa3277: No bootstrap required, opened a new log
I20260809 12:17:19.620106 30878 ts_tablet_manager.cc:1397] T 73ce8294c39c43e987119dab03ba66fc P 10c712c9a0d94e829c9c5c5763aa3277: Time spent bootstrapping tablet: real 0.013s user 0.012s sys 0.000s
I20260809 12:17:19.627691 30877 raft_consensus.cc:348] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565f367d64c347598f718368daa08a26" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36023 } } peers { permanent_uuid: "10c712c9a0d94e829c9c5c5763aa3277" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37655 } } peers { permanent_uuid: "2e535068581c4dbcaca1b42f72e5b911" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42409 } }
I20260809 12:17:19.628307 30877 raft_consensus.cc:374] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:19.628494 30877 raft_consensus.cc:729] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e535068581c4dbcaca1b42f72e5b911, State: Initialized, Role: FOLLOWER
I20260809 12:17:19.629300 30877 consensus_queue.cc:260] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565f367d64c347598f718368daa08a26" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36023 } } peers { permanent_uuid: "10c712c9a0d94e829c9c5c5763aa3277" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37655 } } peers { permanent_uuid: "2e535068581c4dbcaca1b42f72e5b911" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42409 } }
I20260809 12:17:19.631160 30613 heartbeater.cc:503] Master 127.19.183.254:34413 was elected leader, sending a full tablet report...
I20260809 12:17:19.631970 30877 ts_tablet_manager.cc:1428] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911: Time spent starting tablet: real 0.019s user 0.009s sys 0.011s
I20260809 12:17:19.637619 30878 raft_consensus.cc:348] T 73ce8294c39c43e987119dab03ba66fc P 10c712c9a0d94e829c9c5c5763aa3277 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565f367d64c347598f718368daa08a26" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36023 } } peers { permanent_uuid: "10c712c9a0d94e829c9c5c5763aa3277" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37655 } } peers { permanent_uuid: "2e535068581c4dbcaca1b42f72e5b911" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42409 } }
I20260809 12:17:19.638103 30878 raft_consensus.cc:374] T 73ce8294c39c43e987119dab03ba66fc P 10c712c9a0d94e829c9c5c5763aa3277 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:19.638253 30878 raft_consensus.cc:729] T 73ce8294c39c43e987119dab03ba66fc P 10c712c9a0d94e829c9c5c5763aa3277 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10c712c9a0d94e829c9c5c5763aa3277, State: Initialized, Role: FOLLOWER
I20260809 12:17:19.639004 30878 consensus_queue.cc:260] T 73ce8294c39c43e987119dab03ba66fc P 10c712c9a0d94e829c9c5c5763aa3277 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565f367d64c347598f718368daa08a26" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36023 } } peers { permanent_uuid: "10c712c9a0d94e829c9c5c5763aa3277" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37655 } } peers { permanent_uuid: "2e535068581c4dbcaca1b42f72e5b911" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42409 } }
I20260809 12:17:19.644029 30878 ts_tablet_manager.cc:1428] T 73ce8294c39c43e987119dab03ba66fc P 10c712c9a0d94e829c9c5c5763aa3277: Time spent starting tablet: real 0.024s user 0.016s sys 0.004s
W20260809 12:17:19.654747 30385 tablet.cc:2369] T 73ce8294c39c43e987119dab03ba66fc P 10c712c9a0d94e829c9c5c5763aa3277: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:17:19.691233 30818 tablet.cc:2369] T 73ce8294c39c43e987119dab03ba66fc P 565f367d64c347598f718368daa08a26: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:17:19.781872 30615 tablet.cc:2369] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:17:19.869802 30884 raft_consensus.cc:482] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:17:19.870332 30884 raft_consensus.cc:504] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565f367d64c347598f718368daa08a26" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36023 } } peers { permanent_uuid: "10c712c9a0d94e829c9c5c5763aa3277" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37655 } } peers { permanent_uuid: "2e535068581c4dbcaca1b42f72e5b911" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42409 } }
I20260809 12:17:19.872743 30884 leader_election.cc:288] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 565f367d64c347598f718368daa08a26 (127.19.183.195:36023), 10c712c9a0d94e829c9c5c5763aa3277 (127.19.183.193:37655)
I20260809 12:17:19.883112 30872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:19.873320 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:42839 (local address 127.19.183.193:37655)
0809 12:17:19.873668 (+ 348us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:19.873677 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:19.875319 (+ 1642us) server_negotiation.cc:407] Connection header received
0809 12:17:19.875541 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:19.875545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:19.875635 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:19.875729 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:17:19.876977 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.877950 (+ 973us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.880681 (+ 2731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.881674 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.881796 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:19.881849 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:19.882551 (+ 702us) server_negotiation.cc:299] Negotiation successful
0809 12:17:19.882741 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260809 12:17:19.883162 30886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:19.872970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:37655 (local address 127.19.183.194:42839)
0809 12:17:19.874989 (+ 2019us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:19.875027 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:19.875408 (+ 381us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:19.875772 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:19.875780 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:19.875823 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:17:19.876760 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:19.876781 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.878084 (+ 1303us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.878092 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:19.880470 (+ 2378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:19.880486 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.882041 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.882059 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:19.882358 (+ 299us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:19.882396 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 12:17:19.882499 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 12:17:19.882591 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1865,"mutex_wait_us":2,"spinlock_wait_cycles":1792,"thread_start_us":1311,"threads_started":1}
I20260809 12:17:19.883427 30865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:19.873516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:34653 (local address 127.19.183.195:36023)
0809 12:17:19.873772 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:19.873781 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:19.875174 (+ 1393us) server_negotiation.cc:407] Connection header received
0809 12:17:19.875403 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:19.875409 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:19.875507 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:19.875594 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:17:19.876907 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.878050 (+ 1143us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.880827 (+ 2777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.882097 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.882322 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:19.882397 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:19.882998 (+ 601us) server_negotiation.cc:299] Negotiation successful
0809 12:17:19.883149 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260809 12:17:19.883808 30888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:19.872900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:36023 (local address 127.19.183.194:34653)
0809 12:17:19.874914 (+ 2014us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:19.874930 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:19.875181 (+ 251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:19.875643 (+ 462us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:19.875647 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:19.875675 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:17:19.876742 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:19.876761 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.878200 (+ 1439us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.878204 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:19.880642 (+ 2438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:19.880660 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.882518 (+ 1858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.882525 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:19.882817 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:19.882851 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 12:17:19.882962 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 12:17:19.883177 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1767,"mutex_wait_us":24,"spinlock_wait_cycles":7552,"thread_start_us":1559,"threads_started":1}
I20260809 12:17:19.884760 30310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ce8294c39c43e987119dab03ba66fc" candidate_uuid: "2e535068581c4dbcaca1b42f72e5b911" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10c712c9a0d94e829c9c5c5763aa3277" is_pre_election: true
I20260809 12:17:19.885159 30749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ce8294c39c43e987119dab03ba66fc" candidate_uuid: "2e535068581c4dbcaca1b42f72e5b911" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "565f367d64c347598f718368daa08a26" is_pre_election: true
I20260809 12:17:19.885488 30310 raft_consensus.cc:2454] T 73ce8294c39c43e987119dab03ba66fc P 10c712c9a0d94e829c9c5c5763aa3277 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e535068581c4dbcaca1b42f72e5b911 in term 0.
I20260809 12:17:19.885736 30749 raft_consensus.cc:2454] T 73ce8294c39c43e987119dab03ba66fc P 565f367d64c347598f718368daa08a26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e535068581c4dbcaca1b42f72e5b911 in term 0.
I20260809 12:17:19.886435 30440 leader_election.cc:302] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10c712c9a0d94e829c9c5c5763aa3277, 2e535068581c4dbcaca1b42f72e5b911; no voters:
I20260809 12:17:19.886952 30884 raft_consensus.cc:2781] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:17:19.887207 30884 raft_consensus.cc:482] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:17:19.887357 30884 raft_consensus.cc:3037] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:19.890316 30884 raft_consensus.cc:504] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565f367d64c347598f718368daa08a26" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36023 } } peers { permanent_uuid: "10c712c9a0d94e829c9c5c5763aa3277" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37655 } } peers { permanent_uuid: "2e535068581c4dbcaca1b42f72e5b911" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42409 } }
I20260809 12:17:19.891247 30884 leader_election.cc:288] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911 [CANDIDATE]: Term 1 election: Requested vote from peers 565f367d64c347598f718368daa08a26 (127.19.183.195:36023), 10c712c9a0d94e829c9c5c5763aa3277 (127.19.183.193:37655)
I20260809 12:17:19.891785 30749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ce8294c39c43e987119dab03ba66fc" candidate_uuid: "2e535068581c4dbcaca1b42f72e5b911" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "565f367d64c347598f718368daa08a26"
I20260809 12:17:19.891887 30310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "73ce8294c39c43e987119dab03ba66fc" candidate_uuid: "2e535068581c4dbcaca1b42f72e5b911" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10c712c9a0d94e829c9c5c5763aa3277"
I20260809 12:17:19.892082 30749 raft_consensus.cc:3037] T 73ce8294c39c43e987119dab03ba66fc P 565f367d64c347598f718368daa08a26 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:19.892232 30310 raft_consensus.cc:3037] T 73ce8294c39c43e987119dab03ba66fc P 10c712c9a0d94e829c9c5c5763aa3277 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:19.895462 30310 raft_consensus.cc:2454] T 73ce8294c39c43e987119dab03ba66fc P 10c712c9a0d94e829c9c5c5763aa3277 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e535068581c4dbcaca1b42f72e5b911 in term 1.
I20260809 12:17:19.895782 30749 raft_consensus.cc:2454] T 73ce8294c39c43e987119dab03ba66fc P 565f367d64c347598f718368daa08a26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e535068581c4dbcaca1b42f72e5b911 in term 1.
I20260809 12:17:19.895952 30440 leader_election.cc:302] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10c712c9a0d94e829c9c5c5763aa3277, 2e535068581c4dbcaca1b42f72e5b911; no voters:
I20260809 12:17:19.896347 30884 raft_consensus.cc:2781] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:17:19.897133 30884 raft_consensus.cc:686] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911 [term 1 LEADER]: Becoming Leader. State: Replica: 2e535068581c4dbcaca1b42f72e5b911, State: Running, Role: LEADER
I20260809 12:17:19.897801 30884 consensus_queue.cc:237] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565f367d64c347598f718368daa08a26" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36023 } } peers { permanent_uuid: "10c712c9a0d94e829c9c5c5763aa3277" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37655 } } peers { permanent_uuid: "2e535068581c4dbcaca1b42f72e5b911" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42409 } }
I20260809 12:17:19.904222 28920 catalog_manager.cc:5495] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e535068581c4dbcaca1b42f72e5b911 (127.19.183.194). New cstate: current_term: 1 leader_uuid: "2e535068581c4dbcaca1b42f72e5b911" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "565f367d64c347598f718368daa08a26" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36023 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "10c712c9a0d94e829c9c5c5763aa3277" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e535068581c4dbcaca1b42f72e5b911" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42409 } health_report { overall_health: HEALTHY } } }
I20260809 12:17:19.939699 30874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:19.933848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35906 (local address 127.19.183.194:42409)
0809 12:17:19.934098 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:19.934105 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:19.934122 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:17:19.934212 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:19.934215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:19.934329 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:19.934454 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:17:19.934952 (+ 498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.935816 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.937347 (+ 1531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:19.937835 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:19.937933 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:19.938454 (+ 521us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:17:19.938983 (+ 529us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:17:19.939089 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:19.939228 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 12:17:19.939292 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 12:17:19.950670 30310 raft_consensus.cc:1261] T 73ce8294c39c43e987119dab03ba66fc P 10c712c9a0d94e829c9c5c5763aa3277 [term 1 FOLLOWER]: Refusing update from remote peer 2e535068581c4dbcaca1b42f72e5b911: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:17:19.950951 30749 raft_consensus.cc:1261] T 73ce8294c39c43e987119dab03ba66fc P 565f367d64c347598f718368daa08a26 [term 1 FOLLOWER]: Refusing update from remote peer 2e535068581c4dbcaca1b42f72e5b911: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:17:19.952014 30891 consensus_queue.cc:1035] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10c712c9a0d94e829c9c5c5763aa3277" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:17:19.952689 30884 consensus_queue.cc:1035] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911 [LEADER]: Connected to new peer: Peer: permanent_uuid: "565f367d64c347598f718368daa08a26" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:17:19.982946 30900 mvcc.cc:205] Tried to move back new op lower bound from 7316594032425799680 to 7316594032225710080. Current Snapshot: MvccSnapshot[applied={T|T < 7316594032425799680}]
I20260809 12:17:19.992408 28919 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:17:19.992883 28919 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43268:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:17:20.001360 28919 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7852578af3644424bdcd20194612fdac: Sending DeleteTablet for 3 replicas of tablet 73ce8294c39c43e987119dab03ba66fc
I20260809 12:17:20.003173 30275 tablet_service.cc:1514] Processing DeleteTablet for tablet 73ce8294c39c43e987119dab03ba66fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:17:19 UTC) from {username='oryx', principal='oryx/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:32958
I20260809 12:17:20.003894 30721 tablet_service.cc:1514] Processing DeleteTablet for tablet 73ce8294c39c43e987119dab03ba66fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:17:19 UTC) from {username='oryx', principal='oryx/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:48698
I20260809 12:17:20.004820 30511 tablet_service.cc:1514] Processing DeleteTablet for tablet 73ce8294c39c43e987119dab03ba66fc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:17:19 UTC) from {username='oryx', principal='oryx/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:35896
I20260809 12:17:20.014775 30911 tablet_replica.cc:330] T 73ce8294c39c43e987119dab03ba66fc P 10c712c9a0d94e829c9c5c5763aa3277: stopping tablet replica
I20260809 12:17:20.015516 30911 raft_consensus.cc:2229] T 73ce8294c39c43e987119dab03ba66fc P 10c712c9a0d94e829c9c5c5763aa3277 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:17:20.016306 30911 raft_consensus.cc:2258] T 73ce8294c39c43e987119dab03ba66fc P 10c712c9a0d94e829c9c5c5763aa3277 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:17:20.019547 30911 ts_tablet_manager.cc:1905] T 73ce8294c39c43e987119dab03ba66fc P 10c712c9a0d94e829c9c5c5763aa3277: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:17:20.027529 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 28875
I20260809 12:17:20.029392 28960 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:20.040802 30911 ts_tablet_manager.cc:1918] T 73ce8294c39c43e987119dab03ba66fc P 10c712c9a0d94e829c9c5c5763aa3277: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:17:20.041146 30911 log.cc:1196] T 73ce8294c39c43e987119dab03ba66fc P 10c712c9a0d94e829c9c5c5763aa3277: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/73ce8294c39c43e987119dab03ba66fc
I20260809 12:17:20.042354 30911 ts_tablet_manager.cc:1939] T 73ce8294c39c43e987119dab03ba66fc P 10c712c9a0d94e829c9c5c5763aa3277: Deleting consensus metadata
I20260809 12:17:20.054559 30838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:20.007016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50526 (local address 127.19.183.253:40421)
0809 12:17:20.007349 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:20.007358 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:20.023627 (+ 16269us) server_negotiation.cc:407] Connection header received
0809 12:17:20.023734 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:20.023743 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:20.023875 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:20.024032 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:20.026930 (+ 2898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:20.028296 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:20.030679 (+ 2383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:20.031806 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:20.031983 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:20.037292 (+ 5309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:20.037301 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:20.037308 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:20.037355 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:20.043333 (+ 5978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:20.049573 (+ 6240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:20.049582 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:20.049588 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:20.049802 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:20.050831 (+ 1029us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:20.050847 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:20.050853 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:20.051347 (+ 494us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:20.051686 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:20.053634 (+ 1948us) server_negotiation.cc:299] Negotiation successful
0809 12:17:20.053917 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:17:20.060227 30912 tablet_replica.cc:330] T 73ce8294c39c43e987119dab03ba66fc P 565f367d64c347598f718368daa08a26: stopping tablet replica
I20260809 12:17:20.060981 30912 raft_consensus.cc:2229] T 73ce8294c39c43e987119dab03ba66fc P 565f367d64c347598f718368daa08a26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:17:20.063464 30912 raft_consensus.cc:2258] T 73ce8294c39c43e987119dab03ba66fc P 565f367d64c347598f718368daa08a26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:17:20.064036 30913 tablet_replica.cc:330] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911: stopping tablet replica
I20260809 12:17:20.064496 30913 raft_consensus.cc:2229] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:17:20.065372 30913 raft_consensus.cc:2258] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:17:20.067972 30913 ts_tablet_manager.cc:1905] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:17:20.069176 30912 ts_tablet_manager.cc:1905] T 73ce8294c39c43e987119dab03ba66fc P 565f367d64c347598f718368daa08a26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:17:20.080330 30912 ts_tablet_manager.cc:1918] T 73ce8294c39c43e987119dab03ba66fc P 565f367d64c347598f718368daa08a26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:17:20.084654 30916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:20.024812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54218 (local address 127.19.183.252:41811)
0809 12:17:20.062548 (+ 37736us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:20.062553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:20.062568 (+ 15us) server_negotiation.cc:407] Connection header received
0809 12:17:20.062617 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:20.062621 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:20.062693 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:20.065130 (+ 2437us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:20.065324 (+ 194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:20.066884 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:20.069927 (+ 3043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:20.070837 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:20.070980 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:20.074772 (+ 3792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:20.074781 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:20.074788 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:20.074833 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:20.080153 (+ 5320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:20.080953 (+ 800us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:20.080960 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:20.080963 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:20.081143 (+ 180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:20.081763 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:20.081768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:20.081772 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:20.082280 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:20.083792 (+ 1512us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:20.083929 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 12:17:20.084206 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37620,"thread_start_us":37523,"threads_started":1}
I20260809 12:17:20.087203 30913 ts_tablet_manager.cc:1918] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:17:20.087550 30913 log.cc:1196] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/73ce8294c39c43e987119dab03ba66fc
I20260809 12:17:20.088238 30913 ts_tablet_manager.cc:1939] T 73ce8294c39c43e987119dab03ba66fc P 2e535068581c4dbcaca1b42f72e5b911: Deleting consensus metadata
I20260809 12:17:20.089854 30912 log.cc:1196] T 73ce8294c39c43e987119dab03ba66fc P 565f367d64c347598f718368daa08a26: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/73ce8294c39c43e987119dab03ba66fc
I20260809 12:17:20.090590 30912 ts_tablet_manager.cc:1939] T 73ce8294c39c43e987119dab03ba66fc P 565f367d64c347598f718368daa08a26: Deleting consensus metadata
I20260809 12:17:20.256755 28898 catalog_manager.cc:4841] TS 10c712c9a0d94e829c9c5c5763aa3277 (127.19.183.193:37655): tablet 73ce8294c39c43e987119dab03ba66fc (table test-table [id=37c782b20afb4a9ebf92d8682e436dea]) successfully deleted
I20260809 12:17:20.258313 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 28875
I20260809 12:17:20.334651 30838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:20.304369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50542 (local address 127.19.183.253:40421)
0809 12:17:20.304555 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:20.304563 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:20.318293 (+ 13730us) server_negotiation.cc:407] Connection header received
0809 12:17:20.318922 (+ 629us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:20.318932 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:20.319039 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:20.319127 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:20.322266 (+ 3139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:20.323501 (+ 1235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:20.324700 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:20.325335 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:20.325469 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:20.327855 (+ 2386us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:20.327861 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:20.327869 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:20.327900 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:20.331485 (+ 3585us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:20.332450 (+ 965us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:20.332457 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:20.332462 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:20.332625 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:20.333145 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:20.333151 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:20.333153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:20.333470 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:20.333707 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:20.334079 (+ 372us) server_negotiation.cc:299] Negotiation successful
0809 12:17:20.334309 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 12:17:20.334851 30916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:20.304803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54220 (local address 127.19.183.252:41811)
0809 12:17:20.304980 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:20.304986 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:20.312820 (+ 7834us) server_negotiation.cc:407] Connection header received
0809 12:17:20.313046 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:20.313054 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:20.313182 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:20.313305 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:20.316085 (+ 2780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:20.317192 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:20.319405 (+ 2213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:20.320068 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:20.320169 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:20.328293 (+ 8124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:20.328302 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:20.328309 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:20.328363 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:20.331739 (+ 3376us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:20.332527 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:20.332533 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:20.332537 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:20.332694 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:20.333074 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:20.333079 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:20.333083 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:20.333462 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:20.333707 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:20.334283 (+ 576us) server_negotiation.cc:299] Negotiation successful
0809 12:17:20.334511 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:17:20.979261 30930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:20.978278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.19.183.195:60319)
0809 12:17:20.978837 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:20.978992 (+ 155us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":233,"threads_started":1}
W20260809 12:17:20.979754 30816 heartbeater.cc:644] Failed to heartbeat to 127.19.183.254:34413 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
I20260809 12:17:20.983816 30931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:20.983109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.19.183.194:52251)
0809 12:17:20.983537 (+ 428us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:20.983660 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":197,"threads_started":1}
W20260809 12:17:20.984181 30613 heartbeater.cc:644] Failed to heartbeat to 127.19.183.254:34413 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
I20260809 12:17:20.989357 30932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:20.988626 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.19.183.193:50911)
0809 12:17:20.989006 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:20.989162 (+ 156us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":213,"threads_started":1}
W20260809 12:17:20.989773 30374 heartbeater.cc:644] Failed to heartbeat to 127.19.183.254:34413 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
I20260809 12:17:21.502054 30937 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7852578af3644424bdcd20194612fdac)
I20260809 12:17:21.502414 30937 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: false } }
I20260809 12:17:21.502907 30938 raft_consensus.cc:482] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 7852578af3644424bdcd20194612fdac)
I20260809 12:17:21.503600 30938 raft_consensus.cc:504] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: false } }
I20260809 12:17:21.504053 30937 leader_election.cc:288] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7852578af3644424bdcd20194612fdac (127.19.183.254:34413), b399be0c91ba480a873d964b624c404e (127.19.183.253:40421)
I20260809 12:17:21.516534 30941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.504166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.0.0.1:38170)
0809 12:17:21.516003 (+ 11837us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.516217 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":11678,"threads_started":1}
I20260809 12:17:21.518889 30938 leader_election.cc:288] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 7852578af3644424bdcd20194612fdac (127.19.183.254:34413), 5f59b93ddf0e4cc8be0df79193d05d8f (127.19.183.252:41811)
W20260809 12:17:21.519794 30087 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
I20260809 12:17:21.522603 30945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.517928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.0.0.1:38182)
0809 12:17:21.522043 (+ 4115us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.522301 (+ 258us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3923,"thread_start_us":1148,"threads_started":1}
I20260809 12:17:21.522814 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.521181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.0.0.1:38190)
0809 12:17:21.522197 (+ 1016us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.522537 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108}
W20260809 12:17:21.523036 30087 leader_election.cc:334] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7852578af3644424bdcd20194612fdac (127.19.183.254:34413): Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
W20260809 12:17:21.522981 29798 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
I20260809 12:17:21.525146 30945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.524727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.0.0.1:38194)
0809 12:17:21.524907 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.524996 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 12:17:21.525327 29798 leader_election.cc:334] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 7852578af3644424bdcd20194612fdac (127.19.183.254:34413): Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
I20260809 12:17:21.526023 30944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.504480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42014 (local address 127.19.183.253:40421)
0809 12:17:21.517921 (+ 13441us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:21.517927 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:21.517944 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:17:21.518010 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:21.518018 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:21.518118 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:21.518389 (+ 271us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:17:21.519329 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.520764 (+ 1435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.523576 (+ 2812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.524816 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.524984 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:21.525052 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:21.525666 (+ 614us) server_negotiation.cc:299] Negotiation successful
0809 12:17:21.525769 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":13330,"thread_start_us":13154,"threads_started":1}
I20260809 12:17:21.526057 30941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.504401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:40421 (local address 127.0.0.1:42014)
0809 12:17:21.517121 (+ 12720us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.517142 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:21.517372 (+ 230us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:21.518515 (+ 1143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:21.518523 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:21.518562 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:17:21.519015 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:21.519031 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.520985 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.520994 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:21.523352 (+ 2358us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:21.523365 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.524981 (+ 1616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.524987 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:21.525173 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:21.525209 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 12:17:21.525631 (+ 422us) client_negotiation.cc:240] Negotiation successful
0809 12:17:21.525771 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12596,"mutex_wait_us":19,"thread_start_us":12338,"threads_started":1}
I20260809 12:17:21.526839 29830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b399be0c91ba480a873d964b624c404e" is_pre_election: true
I20260809 12:17:21.527168 29830 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f59b93ddf0e4cc8be0df79193d05d8f in term 1.
I20260809 12:17:21.527683 30086 leader_election.cc:302] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f59b93ddf0e4cc8be0df79193d05d8f, b399be0c91ba480a873d964b624c404e; no voters: 7852578af3644424bdcd20194612fdac
I20260809 12:17:21.528143 30937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 12:17:21.528326 30949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.505381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:41811 (local address 127.0.0.1:41804)
0809 12:17:21.519324 (+ 13943us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.519344 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:21.519540 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:21.521415 (+ 1875us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:21.521420 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:21.521448 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:17:21.522062 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:21.522078 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.524127 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.524133 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:21.526244 (+ 2111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:21.526261 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.527548 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.527554 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:21.527698 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:21.527721 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 12:17:21.527794 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 12:17:21.527909 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":13823,"mutex_wait_us":49,"thread_start_us":16286,"threads_started":1}
I20260809 12:17:21.528321 30937 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 1 FOLLOWER]: Starting leader election (detected failure of leader 7852578af3644424bdcd20194612fdac)
I20260809 12:17:21.528445 30937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 1 FOLLOWER]: Advancing to term 2
I20260809 12:17:21.528466 30948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.505934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41804 (local address 127.19.183.252:41811)
0809 12:17:21.521045 (+ 15111us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:21.521053 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:21.521072 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:17:21.521143 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:21.521148 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:21.521219 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:21.521399 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:17:21.522419 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.523987 (+ 1568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.526409 (+ 2422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.527384 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.527552 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:21.527615 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:21.527858 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 12:17:21.527982 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":422,"server-negotiator.queue_time_us":14955,"thread_start_us":14836,"threads_started":1}
I20260809 12:17:21.529094 30115 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b399be0c91ba480a873d964b624c404e" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" is_pre_election: true
I20260809 12:17:21.531301 30937 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 2 FOLLOWER]: Starting leader election with config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: false } }
I20260809 12:17:21.531845 30115 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b399be0c91ba480a873d964b624c404e in current term 2: Already voted for candidate 5f59b93ddf0e4cc8be0df79193d05d8f in this term.
I20260809 12:17:21.532012 30937 leader_election.cc:288] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [CANDIDATE]: Term 2 election: Requested vote from peers 7852578af3644424bdcd20194612fdac (127.19.183.254:34413), b399be0c91ba480a873d964b624c404e (127.19.183.253:40421)
I20260809 12:17:21.532554 30941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.532072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.0.0.1:38198)
0809 12:17:21.532290 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.532381 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260809 12:17:21.532634 29830 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b399be0c91ba480a873d964b624c404e"
I20260809 12:17:21.532898 29830 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [term 1 FOLLOWER]: Advancing to term 2
I20260809 12:17:21.532964 29794 leader_election.cc:302] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b399be0c91ba480a873d964b624c404e; no voters: 5f59b93ddf0e4cc8be0df79193d05d8f, 7852578af3644424bdcd20194612fdac
I20260809 12:17:21.533564 30941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.533222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.0.0.1:38204)
0809 12:17:21.533405 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.533475 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
W20260809 12:17:21.533731 30087 leader_election.cc:334] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 7852578af3644424bdcd20194612fdac (127.19.183.254:34413): Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
I20260809 12:17:21.535256 29830 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f59b93ddf0e4cc8be0df79193d05d8f in term 2.
I20260809 12:17:21.535413 30938 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 12:17:21.535887 30086 leader_election.cc:302] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5f59b93ddf0e4cc8be0df79193d05d8f, b399be0c91ba480a873d964b624c404e; no voters: 7852578af3644424bdcd20194612fdac
I20260809 12:17:21.536168 30937 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 2 FOLLOWER]: Leader election won for term 2
I20260809 12:17:21.536571 30937 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [term 2 LEADER]: Becoming Leader. State: Replica: 5f59b93ddf0e4cc8be0df79193d05d8f, State: Running, Role: LEADER
I20260809 12:17:21.536962 30937 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: false } }
I20260809 12:17:21.538652 30954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f59b93ddf0e4cc8be0df79193d05d8f. Latest consensus state: current_term: 2 leader_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: false } } }
I20260809 12:17:21.539117 30954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:21.539848 30957 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:17:21.543792 30957 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:17:21.544585 30957 catalog_manager.cc:1259] Loaded cluster ID: e474dd64bd7a4984a89c538000646418
I20260809 12:17:21.544678 30957 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:17:21.545776 30957 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:17:21.546568 30957 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f: Loaded TSK: 0
I20260809 12:17:21.547420 30957 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:17:21.563910 30101 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54220:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:17:21.570955 30101 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:17:21.574687 30941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.574283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.0.0.1:38224)
0809 12:17:21.574472 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.574542 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 12:17:21.574915 29830 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [term 2 FOLLOWER]: Refusing update from remote peer 5f59b93ddf0e4cc8be0df79193d05d8f: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 12:17:21.575691 30941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.575342 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.0.0.1:38226)
0809 12:17:21.575481 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.575556 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 12:17:21.575812 30954 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [LEADER]: Connected to new peer: Peer: permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260809 12:17:21.576107 30087 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f -> Peer 7852578af3644424bdcd20194612fdac (127.19.183.254:34413): Couldn't send request to peer 7852578af3644424bdcd20194612fdac. Status: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 12:17:21.578696 30938 sys_catalog.cc:455] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5f59b93ddf0e4cc8be0df79193d05d8f. Latest consensus state: current_term: 2 leader_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: false } } }
I20260809 12:17:21.579025 30938 sys_catalog.cc:458] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:17:21.580454 30954 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: false } } }
I20260809 12:17:21.580754 30954 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:21.581043 30938 sys_catalog.cc:455] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" committed_config { opid_index: 8 OBSOLETE_local: true peers { permanent_uuid: "7852578af3644424bdcd20194612fdac" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 34413 } } peers { permanent_uuid: "b399be0c91ba480a873d964b624c404e" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 40421 } attrs { promote: false } } peers { permanent_uuid: "5f59b93ddf0e4cc8be0df79193d05d8f" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 41811 } attrs { promote: false } } }
I20260809 12:17:21.581328 30938 sys_catalog.cc:458] T 00000000000000000000000000000000 P b399be0c91ba480a873d964b624c404e [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:17:21.596798 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.590174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:36023 (local address 127.0.0.1:37424)
0809 12:17:21.590875 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.590898 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:21.591146 (+ 248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:21.591719 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:21.591724 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:21.591750 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:17:21.592024 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:21.592034 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.593246 (+ 1212us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.593252 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:21.594892 (+ 1640us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:21.594904 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.596049 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.596055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:21.596222 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:21.596246 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 12:17:21.596351 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 12:17:21.596489 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561}
I20260809 12:17:21.596820 30971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.590317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37424 (local address 127.19.183.195:36023)
0809 12:17:21.590878 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:21.590882 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:21.591068 (+ 186us) server_negotiation.cc:407] Connection header received
0809 12:17:21.591276 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:21.591280 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:21.591487 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:21.592151 (+ 664us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:17:21.592284 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.593083 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.595078 (+ 1995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.595852 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.596017 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:21.596092 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:21.596382 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 12:17:21.596513 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":301,"threads_started":1}
I20260809 12:17:21.597891 30721 tablet_service.cc:1467] Processing CreateTablet for tablet e4a65d77057c4bf593e546dc16a6d235 (DEFAULT_TABLE table=test-table [id=caf3e2204d52425fbeb20d3f43e7e373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:17:21.598474 30721 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4a65d77057c4bf593e546dc16a6d235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:21.605139 30941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.589337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:37655 (local address 127.0.0.1:52712)
0809 12:17:21.589508 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.589527 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:21.589655 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:21.599381 (+ 9726us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:21.599388 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:21.599419 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:17:21.599580 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:21.599589 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.600668 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.600677 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:21.602451 (+ 1774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:21.602468 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.604363 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.604370 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:21.604605 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:21.604639 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 12:17:21.604708 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 12:17:21.604829 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 12:17:21.605981 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.589453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52712 (local address 127.19.183.193:37655)
0809 12:17:21.599030 (+ 9577us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:21.599035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:21.599068 (+ 33us) server_negotiation.cc:407] Connection header received
0809 12:17:21.599135 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:21.599138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:21.599191 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:21.599321 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:17:21.599747 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.600512 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.602635 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.604173 (+ 1538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.605389 (+ 1216us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:21.605464 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:21.605520 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 12:17:21.605670 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9476,"thread_start_us":9397,"threads_started":1}
I20260809 12:17:21.606091 30974 tablet_bootstrap.cc:492] T e4a65d77057c4bf593e546dc16a6d235 P 565f367d64c347598f718368daa08a26: Bootstrap starting.
I20260809 12:17:21.606962 30275 tablet_service.cc:1467] Processing CreateTablet for tablet e4a65d77057c4bf593e546dc16a6d235 (DEFAULT_TABLE table=test-table [id=caf3e2204d52425fbeb20d3f43e7e373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:17:21.607425 30275 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4a65d77057c4bf593e546dc16a6d235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:21.608230 30974 tablet_bootstrap.cc:654] T e4a65d77057c4bf593e546dc16a6d235 P 565f367d64c347598f718368daa08a26: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:21.610304 30974 tablet_bootstrap.cc:492] T e4a65d77057c4bf593e546dc16a6d235 P 565f367d64c347598f718368daa08a26: No bootstrap required, opened a new log
I20260809 12:17:21.610502 30974 ts_tablet_manager.cc:1397] T e4a65d77057c4bf593e546dc16a6d235 P 565f367d64c347598f718368daa08a26: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 12:17:21.610570 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.590306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:42409 (local address 127.0.0.1:38192)
0809 12:17:21.597148 (+ 6842us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.597167 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:21.597252 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:21.604379 (+ 7127us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:21.604385 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:21.604418 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:17:21.604611 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:21.604621 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.606411 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.606417 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:21.608166 (+ 1749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:21.608183 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.609707 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.609712 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:21.609924 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:21.609969 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 12:17:21.610071 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 12:17:21.610230 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6728,"thread_start_us":13772,"threads_started":1}
I20260809 12:17:21.610605 30973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.591476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38192 (local address 127.19.183.194:42409)
0809 12:17:21.603787 (+ 12311us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:21.603793 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:21.603813 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:17:21.603880 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:21.603885 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:21.603953 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:21.604755 (+ 802us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:17:21.605006 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.606240 (+ 1234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.608376 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.609542 (+ 1166us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.609730 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:21.609810 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:21.610134 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 12:17:21.610278 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12148,"thread_start_us":12035,"threads_started":1}
I20260809 12:17:21.611492 30511 tablet_service.cc:1467] Processing CreateTablet for tablet e4a65d77057c4bf593e546dc16a6d235 (DEFAULT_TABLE table=test-table [id=caf3e2204d52425fbeb20d3f43e7e373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:17:21.611938 30511 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4a65d77057c4bf593e546dc16a6d235. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:21.612254 30974 raft_consensus.cc:348] T e4a65d77057c4bf593e546dc16a6d235 P 565f367d64c347598f718368daa08a26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c712c9a0d94e829c9c5c5763aa3277" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37655 } } peers { permanent_uuid: "2e535068581c4dbcaca1b42f72e5b911" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42409 } } peers { permanent_uuid: "565f367d64c347598f718368daa08a26" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36023 } }
I20260809 12:17:21.612442 30974 raft_consensus.cc:374] T e4a65d77057c4bf593e546dc16a6d235 P 565f367d64c347598f718368daa08a26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:21.612500 30974 raft_consensus.cc:729] T e4a65d77057c4bf593e546dc16a6d235 P 565f367d64c347598f718368daa08a26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 565f367d64c347598f718368daa08a26, State: Initialized, Role: FOLLOWER
I20260809 12:17:21.612735 30974 consensus_queue.cc:260] T e4a65d77057c4bf593e546dc16a6d235 P 565f367d64c347598f718368daa08a26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c712c9a0d94e829c9c5c5763aa3277" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37655 } } peers { permanent_uuid: "2e535068581c4dbcaca1b42f72e5b911" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42409 } } peers { permanent_uuid: "565f367d64c347598f718368daa08a26" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36023 } }
I20260809 12:17:21.613507 30974 ts_tablet_manager.cc:1428] T e4a65d77057c4bf593e546dc16a6d235 P 565f367d64c347598f718368daa08a26: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 12:17:21.618265 30814 heartbeater.cc:503] Master 127.19.183.252:41811 was elected leader, sending a full tablet report...
I20260809 12:17:21.622121 30981 tablet_bootstrap.cc:492] T e4a65d77057c4bf593e546dc16a6d235 P 10c712c9a0d94e829c9c5c5763aa3277: Bootstrap starting.
I20260809 12:17:21.622995 30982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.614157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.19.183.195:59061)
0809 12:17:21.621195 (+ 7038us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.621332 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":6909,"thread_start_us":6788,"threads_started":1}
I20260809 12:17:21.623829 30981 tablet_bootstrap.cc:654] T e4a65d77057c4bf593e546dc16a6d235 P 10c712c9a0d94e829c9c5c5763aa3277: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:21.624334 30982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.623872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.19.183.195:60683)
0809 12:17:21.624087 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.624204 (+ 117us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113}
I20260809 12:17:21.626009 30981 tablet_bootstrap.cc:492] T e4a65d77057c4bf593e546dc16a6d235 P 10c712c9a0d94e829c9c5c5763aa3277: No bootstrap required, opened a new log
I20260809 12:17:21.626154 30981 ts_tablet_manager.cc:1397] T e4a65d77057c4bf593e546dc16a6d235 P 10c712c9a0d94e829c9c5c5763aa3277: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.003s
I20260809 12:17:21.626828 30981 raft_consensus.cc:348] T e4a65d77057c4bf593e546dc16a6d235 P 10c712c9a0d94e829c9c5c5763aa3277 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c712c9a0d94e829c9c5c5763aa3277" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37655 } } peers { permanent_uuid: "2e535068581c4dbcaca1b42f72e5b911" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42409 } } peers { permanent_uuid: "565f367d64c347598f718368daa08a26" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36023 } }
I20260809 12:17:21.627138 30981 raft_consensus.cc:374] T e4a65d77057c4bf593e546dc16a6d235 P 10c712c9a0d94e829c9c5c5763aa3277 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:21.627261 30983 tablet_bootstrap.cc:492] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911: Bootstrap starting.
I20260809 12:17:21.627283 30981 raft_consensus.cc:729] T e4a65d77057c4bf593e546dc16a6d235 P 10c712c9a0d94e829c9c5c5763aa3277 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10c712c9a0d94e829c9c5c5763aa3277, State: Initialized, Role: FOLLOWER
I20260809 12:17:21.627631 30981 consensus_queue.cc:260] T e4a65d77057c4bf593e546dc16a6d235 P 10c712c9a0d94e829c9c5c5763aa3277 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c712c9a0d94e829c9c5c5763aa3277" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37655 } } peers { permanent_uuid: "2e535068581c4dbcaca1b42f72e5b911" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42409 } } peers { permanent_uuid: "565f367d64c347598f718368daa08a26" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36023 } }
I20260809 12:17:21.628508 30981 ts_tablet_manager.cc:1428] T e4a65d77057c4bf593e546dc16a6d235 P 10c712c9a0d94e829c9c5c5763aa3277: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 12:17:21.629288 30983 tablet_bootstrap.cc:654] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:21.629617 30986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.628679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.19.183.193:36675)
0809 12:17:21.629242 (+ 563us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.629407 (+ 165us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":437,"thread_start_us":282,"threads_started":1}
I20260809 12:17:21.630563 30373 heartbeater.cc:503] Master 127.19.183.252:41811 was elected leader, sending a full tablet report...
I20260809 12:17:21.631286 30986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.630245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.19.183.193:60253)
0809 12:17:21.631067 (+ 822us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.631160 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":724}
I20260809 12:17:21.633275 30983 tablet_bootstrap.cc:492] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911: No bootstrap required, opened a new log
I20260809 12:17:21.633451 30983 ts_tablet_manager.cc:1397] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260809 12:17:21.634223 30983 raft_consensus.cc:348] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c712c9a0d94e829c9c5c5763aa3277" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37655 } } peers { permanent_uuid: "2e535068581c4dbcaca1b42f72e5b911" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42409 } } peers { permanent_uuid: "565f367d64c347598f718368daa08a26" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36023 } }
I20260809 12:17:21.634426 30983 raft_consensus.cc:374] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:21.634528 30983 raft_consensus.cc:729] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e535068581c4dbcaca1b42f72e5b911, State: Initialized, Role: FOLLOWER
I20260809 12:17:21.634840 30983 consensus_queue.cc:260] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c712c9a0d94e829c9c5c5763aa3277" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37655 } } peers { permanent_uuid: "2e535068581c4dbcaca1b42f72e5b911" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42409 } } peers { permanent_uuid: "565f367d64c347598f718368daa08a26" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36023 } }
I20260809 12:17:21.635751 30983 ts_tablet_manager.cc:1428] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 12:17:21.637455 30610 heartbeater.cc:503] Master 127.19.183.252:41811 was elected leader, sending a full tablet report...
I20260809 12:17:21.640556 30989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.635962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.19.183.194:35441)
0809 12:17:21.640034 (+ 4072us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.640292 (+ 258us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3959,"thread_start_us":3847,"threads_started":1}
I20260809 12:17:21.641386 30989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.641059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.19.183.194:47809)
0809 12:17:21.641217 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.641292 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260809 12:17:21.649881 30988 raft_consensus.cc:482] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:17:21.650048 30988 raft_consensus.cc:504] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c712c9a0d94e829c9c5c5763aa3277" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37655 } } peers { permanent_uuid: "2e535068581c4dbcaca1b42f72e5b911" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42409 } } peers { permanent_uuid: "565f367d64c347598f718368daa08a26" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36023 } }
I20260809 12:17:21.650789 30988 leader_election.cc:288] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 10c712c9a0d94e829c9c5c5763aa3277 (127.19.183.193:37655), 565f367d64c347598f718368daa08a26 (127.19.183.195:36023)
I20260809 12:17:21.651414 30310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a65d77057c4bf593e546dc16a6d235" candidate_uuid: "2e535068581c4dbcaca1b42f72e5b911" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10c712c9a0d94e829c9c5c5763aa3277" is_pre_election: true
I20260809 12:17:21.651414 30749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a65d77057c4bf593e546dc16a6d235" candidate_uuid: "2e535068581c4dbcaca1b42f72e5b911" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "565f367d64c347598f718368daa08a26" is_pre_election: true
I20260809 12:17:21.651676 30310 raft_consensus.cc:2454] T e4a65d77057c4bf593e546dc16a6d235 P 10c712c9a0d94e829c9c5c5763aa3277 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e535068581c4dbcaca1b42f72e5b911 in term 0.
I20260809 12:17:21.651679 30749 raft_consensus.cc:2454] T e4a65d77057c4bf593e546dc16a6d235 P 565f367d64c347598f718368daa08a26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e535068581c4dbcaca1b42f72e5b911 in term 0.
I20260809 12:17:21.652202 30440 leader_election.cc:302] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 10c712c9a0d94e829c9c5c5763aa3277, 2e535068581c4dbcaca1b42f72e5b911, 565f367d64c347598f718368daa08a26; no voters:
I20260809 12:17:21.652514 30988 raft_consensus.cc:2781] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:17:21.652657 30988 raft_consensus.cc:482] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:17:21.652746 30988 raft_consensus.cc:3037] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:21.655130 30988 raft_consensus.cc:504] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c712c9a0d94e829c9c5c5763aa3277" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37655 } } peers { permanent_uuid: "2e535068581c4dbcaca1b42f72e5b911" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42409 } } peers { permanent_uuid: "565f367d64c347598f718368daa08a26" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36023 } }
I20260809 12:17:21.655709 30988 leader_election.cc:288] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911 [CANDIDATE]: Term 1 election: Requested vote from peers 10c712c9a0d94e829c9c5c5763aa3277 (127.19.183.193:37655), 565f367d64c347598f718368daa08a26 (127.19.183.195:36023)
I20260809 12:17:21.656250 30310 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a65d77057c4bf593e546dc16a6d235" candidate_uuid: "2e535068581c4dbcaca1b42f72e5b911" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "10c712c9a0d94e829c9c5c5763aa3277"
I20260809 12:17:21.656314 30749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4a65d77057c4bf593e546dc16a6d235" candidate_uuid: "2e535068581c4dbcaca1b42f72e5b911" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "565f367d64c347598f718368daa08a26"
I20260809 12:17:21.656479 30310 raft_consensus.cc:3037] T e4a65d77057c4bf593e546dc16a6d235 P 10c712c9a0d94e829c9c5c5763aa3277 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:21.656500 30749 raft_consensus.cc:3037] T e4a65d77057c4bf593e546dc16a6d235 P 565f367d64c347598f718368daa08a26 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:21.659375 30749 raft_consensus.cc:2454] T e4a65d77057c4bf593e546dc16a6d235 P 565f367d64c347598f718368daa08a26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e535068581c4dbcaca1b42f72e5b911 in term 1.
I20260809 12:17:21.659375 30310 raft_consensus.cc:2454] T e4a65d77057c4bf593e546dc16a6d235 P 10c712c9a0d94e829c9c5c5763aa3277 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e535068581c4dbcaca1b42f72e5b911 in term 1.
I20260809 12:17:21.659791 30440 leader_election.cc:302] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 10c712c9a0d94e829c9c5c5763aa3277, 2e535068581c4dbcaca1b42f72e5b911; no voters:
I20260809 12:17:21.660076 30988 raft_consensus.cc:2781] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:17:21.660563 30988 raft_consensus.cc:686] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911 [term 1 LEADER]: Becoming Leader. State: Replica: 2e535068581c4dbcaca1b42f72e5b911, State: Running, Role: LEADER
I20260809 12:17:21.660861 30988 consensus_queue.cc:237] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c712c9a0d94e829c9c5c5763aa3277" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37655 } } peers { permanent_uuid: "2e535068581c4dbcaca1b42f72e5b911" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42409 } } peers { permanent_uuid: "565f367d64c347598f718368daa08a26" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36023 } }
I20260809 12:17:21.661624 30989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.661183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.19.183.194:44617)
0809 12:17:21.661341 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.661435 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 12:17:21.663789 30101 catalog_manager.cc:5495] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e535068581c4dbcaca1b42f72e5b911 (127.19.183.194). New cstate: current_term: 1 leader_uuid: "2e535068581c4dbcaca1b42f72e5b911" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "10c712c9a0d94e829c9c5c5763aa3277" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e535068581c4dbcaca1b42f72e5b911" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 42409 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "565f367d64c347598f718368daa08a26" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36023 } health_report { overall_health: UNKNOWN } } }
I20260809 12:17:21.692548 30973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.688142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38204 (local address 127.19.183.194:42409)
0809 12:17:21.688332 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:21.688337 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:21.688356 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:17:21.688413 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:21.688416 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:21.688489 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:21.688564 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:17:21.688887 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.689593 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.691030 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.691493 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.691584 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:21.692005 (+ 421us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:17:21.692114 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:17:21.692183 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:21.692285 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 12:17:21.692342 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 12:17:21.695858 30749 raft_consensus.cc:1261] T e4a65d77057c4bf593e546dc16a6d235 P 565f367d64c347598f718368daa08a26 [term 1 FOLLOWER]: Refusing update from remote peer 2e535068581c4dbcaca1b42f72e5b911: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:17:21.695983 30310 raft_consensus.cc:1261] T e4a65d77057c4bf593e546dc16a6d235 P 10c712c9a0d94e829c9c5c5763aa3277 [term 1 FOLLOWER]: Refusing update from remote peer 2e535068581c4dbcaca1b42f72e5b911: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:17:21.696575 30991 consensus_queue.cc:1035] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911 [LEADER]: Connected to new peer: Peer: permanent_uuid: "565f367d64c347598f718368daa08a26" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 36023 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:17:21.696943 30988 consensus_queue.cc:1035] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911 [LEADER]: Connected to new peer: Peer: permanent_uuid: "10c712c9a0d94e829c9c5c5763aa3277" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:17:21.699527 30982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.699009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.19.183.195:60397)
0809 12:17:21.699197 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.699319 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 12:17:21.699848 30986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.699341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.19.183.193:44095)
0809 12:17:21.699528 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.699666 (+ 138us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 12:17:21.703868 30995 mvcc.cc:205] Tried to move back new op lower bound from 7316594039579594752 to 7316594039444987904. Current Snapshot: MvccSnapshot[applied={T|T < 7316594039579594752}]
I20260809 12:17:21.704190 30989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.703674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.19.183.194:58243)
0809 12:17:21.703910 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.704013 (+ 103us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124}
I20260809 12:17:21.705746 30982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.705005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.19.183.195:44941)
0809 12:17:21.705495 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.705582 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379}
I20260809 12:17:21.710282 30101 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:17:21.710573 30101 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54220:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:17:21.711225 30999 mvcc.cc:205] Tried to move back new op lower bound from 7316594039579594752 to 7316594039444987904. Current Snapshot: MvccSnapshot[applied={T|T < 7316594039579594752}]
I20260809 12:17:21.719169 30101 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f: Sending DeleteTablet for 3 replicas of tablet e4a65d77057c4bf593e546dc16a6d235
I20260809 12:17:21.720542 30720 tablet_service.cc:1514] Processing DeleteTablet for tablet e4a65d77057c4bf593e546dc16a6d235 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:17:21 UTC) from {username='oryx', principal='oryx/127.19.183.252@KRBTEST.COM'} at 127.0.0.1:37424
I20260809 12:17:21.721038 30721 tablet_service.cc:1514] Processing DeleteTablet for tablet e4a65d77057c4bf593e546dc16a6d235 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:17:21 UTC) from {username='oryx', principal='oryx/127.19.183.252@KRBTEST.COM'} at 127.0.0.1:37424
I20260809 12:17:21.721328 30511 tablet_service.cc:1514] Processing DeleteTablet for tablet e4a65d77057c4bf593e546dc16a6d235 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:17:21 UTC) from {username='oryx', principal='oryx/127.19.183.252@KRBTEST.COM'} at 127.0.0.1:38192
I20260809 12:17:21.721465 30275 tablet_service.cc:1514] Processing DeleteTablet for tablet e4a65d77057c4bf593e546dc16a6d235 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:17:21 UTC) from {username='oryx', principal='oryx/127.19.183.252@KRBTEST.COM'} at 127.0.0.1:52712
I20260809 12:17:21.722187 30989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.720767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.19.183.194:55135)
0809 12:17:21.721883 (+ 1116us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.721974 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":982}
I20260809 12:17:21.722568 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 30065
I20260809 12:17:21.732153 31010 tablet_replica.cc:330] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911: stopping tablet replica
I20260809 12:17:21.732340 31010 raft_consensus.cc:2229] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:17:21.732476 30986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.731916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.19.183.193:53217)
0809 12:17:21.732137 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:21.732264 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 12:17:21.732786 31010 raft_consensus.cc:2258] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:17:21.736338 31009 tablet_replica.cc:330] T e4a65d77057c4bf593e546dc16a6d235 P 565f367d64c347598f718368daa08a26: stopping tablet replica
I20260809 12:17:21.736418 31012 tablet_replica.cc:330] T e4a65d77057c4bf593e546dc16a6d235 P 10c712c9a0d94e829c9c5c5763aa3277: stopping tablet replica
I20260809 12:17:21.736534 31009 raft_consensus.cc:2229] T e4a65d77057c4bf593e546dc16a6d235 P 565f367d64c347598f718368daa08a26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:17:21.736611 31012 raft_consensus.cc:2229] T e4a65d77057c4bf593e546dc16a6d235 P 10c712c9a0d94e829c9c5c5763aa3277 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:17:21.736837 31009 raft_consensus.cc:2258] T e4a65d77057c4bf593e546dc16a6d235 P 565f367d64c347598f718368daa08a26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:17:21.737449 31012 raft_consensus.cc:2258] T e4a65d77057c4bf593e546dc16a6d235 P 10c712c9a0d94e829c9c5c5763aa3277 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:17:21.738314 31009 ts_tablet_manager.cc:1905] T e4a65d77057c4bf593e546dc16a6d235 P 565f367d64c347598f718368daa08a26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:17:21.738466 31010 ts_tablet_manager.cc:1905] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:17:21.739861 31012 ts_tablet_manager.cc:1905] T e4a65d77057c4bf593e546dc16a6d235 P 10c712c9a0d94e829c9c5c5763aa3277: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:17:21.745170 31010 ts_tablet_manager.cc:1918] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:17:21.745325 31010 log.cc:1196] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/e4a65d77057c4bf593e546dc16a6d235
I20260809 12:17:21.745718 31012 ts_tablet_manager.cc:1918] T e4a65d77057c4bf593e546dc16a6d235 P 10c712c9a0d94e829c9c5c5763aa3277: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:17:21.745817 31010 ts_tablet_manager.cc:1939] T e4a65d77057c4bf593e546dc16a6d235 P 2e535068581c4dbcaca1b42f72e5b911: Deleting consensus metadata
I20260809 12:17:21.745884 31012 log.cc:1196] T e4a65d77057c4bf593e546dc16a6d235 P 10c712c9a0d94e829c9c5c5763aa3277: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/e4a65d77057c4bf593e546dc16a6d235
I20260809 12:17:21.746335 31012 ts_tablet_manager.cc:1939] T e4a65d77057c4bf593e546dc16a6d235 P 10c712c9a0d94e829c9c5c5763aa3277: Deleting consensus metadata
I20260809 12:17:21.746945 30083 catalog_manager.cc:4841] TS 2e535068581c4dbcaca1b42f72e5b911 (127.19.183.194:42409): tablet e4a65d77057c4bf593e546dc16a6d235 (table test-table [id=caf3e2204d52425fbeb20d3f43e7e373]) successfully deleted
I20260809 12:17:21.747915 30082 catalog_manager.cc:4841] TS 10c712c9a0d94e829c9c5c5763aa3277 (127.19.183.193:37655): tablet e4a65d77057c4bf593e546dc16a6d235 (table test-table [id=caf3e2204d52425fbeb20d3f43e7e373]) successfully deleted
I20260809 12:17:21.748555 31009 ts_tablet_manager.cc:1918] T e4a65d77057c4bf593e546dc16a6d235 P 565f367d64c347598f718368daa08a26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:17:21.748713 31009 log.cc:1196] T e4a65d77057c4bf593e546dc16a6d235 P 565f367d64c347598f718368daa08a26: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/e4a65d77057c4bf593e546dc16a6d235
I20260809 12:17:21.749215 31009 ts_tablet_manager.cc:1939] T e4a65d77057c4bf593e546dc16a6d235 P 565f367d64c347598f718368daa08a26: Deleting consensus metadata
I20260809 12:17:21.750404 30086 catalog_manager.cc:4841] TS 565f367d64c347598f718368daa08a26 (127.19.183.195:36023): tablet e4a65d77057c4bf593e546dc16a6d235 (table test-table [id=caf3e2204d52425fbeb20d3f43e7e373]) successfully deleted
W20260809 12:17:21.751121 30086 catalog_manager.cc:4808] TS 565f367d64c347598f718368daa08a26 (127.19.183.195:36023): delete failed for tablet e4a65d77057c4bf593e546dc16a6d235 because the tablet was not found. No further retry: Not found: Tablet not found: e4a65d77057c4bf593e546dc16a6d235
I20260809 12:17:21.754601 30970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.723551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52714 (local address 127.19.183.193:37655)
0809 12:17:21.723784 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:21.723791 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:21.731553 (+ 7762us) server_negotiation.cc:407] Connection header received
0809 12:17:21.733191 (+ 1638us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:21.733200 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:21.733303 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:21.733391 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:21.739004 (+ 5613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.740111 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.741973 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.742641 (+ 668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.742809 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:21.746565 (+ 3756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:21.746584 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:21.746610 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:21.746662 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:21.751936 (+ 5274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:21.752819 (+ 883us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:21.752826 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:21.752829 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:21.752954 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:21.753322 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:21.753325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:21.753327 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:21.753680 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:21.753885 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:21.754220 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 12:17:21.754385 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260809 12:17:21.755293 30366 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:21.865383 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 30065
I20260809 12:17:21.894503 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 30392
I20260809 12:17:21.908265 30973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:21.895336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38208 (local address 127.19.183.194:42409)
0809 12:17:21.895528 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:21.895536 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:21.895556 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:17:21.895771 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:21.895776 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:21.895856 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:21.895973 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:21.897366 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.898069 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.899463 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:21.899902 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:21.900035 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:21.902581 (+ 2546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:21.902596 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:21.902612 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:21.902648 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:21.905420 (+ 2772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:21.906047 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:21.906055 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:21.906057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:21.906183 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:21.906555 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:21.906559 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:21.906562 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:21.907033 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:21.907362 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:21.907701 (+ 339us) server_negotiation.cc:299] Negotiation successful
0809 12:17:21.907937 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 12:17:21.908927 30597 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:22.026355 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:22.025879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.0.0.1:38242)
0809 12:17:22.026061 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:22.026175 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 12:17:22.027696 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:22.027332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:34413 (local address 127.0.0.1:38258)
0809 12:17:22.027482 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:22.027561 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:34413: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 12:17:22.034409 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 30392
I20260809 12:17:22.066911 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 30627
I20260809 12:17:22.079948 30971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:22.067732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37428 (local address 127.19.183.195:36023)
0809 12:17:22.067930 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:22.067934 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:22.067953 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:17:22.068079 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:22.068083 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:22.068150 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:22.068281 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:22.069675 (+ 1394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:22.070337 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:22.071659 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:22.072092 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:22.072190 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:22.074640 (+ 2450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:22.074652 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:22.074669 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:22.074695 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:22.077496 (+ 2801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:22.078209 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:22.078215 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:22.078218 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:22.078331 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:22.078637 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:22.078640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:22.078642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:22.079028 (+ 386us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:22.079264 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:22.079492 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 12:17:22.079690 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 12:17:22.080610 30806 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:22.173367 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 30627
I20260809 12:17:22.196107 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 29596
I20260809 12:17:22.197212 29861 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:22.313448 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 29596
W20260809 12:17:22.343117 30086 connection.cc:535] client connection to 127.19.183.253:40421 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 12:17:22.343506 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 29752
I20260809 12:17:22.344488 30940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:22.344049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:40421 (local address 127.0.0.1:42018)
0809 12:17:22.344226 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:22.344306 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:40421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260809 12:17:22.344554 30138 generic_service.cc:196] Checking for leaks (request via RPC)
W20260809 12:17:22.344771 30086 consensus_peers.cc:487] T 00000000000000000000000000000000 P 5f59b93ddf0e4cc8be0df79193d05d8f -> Peer b399be0c91ba480a873d964b624c404e (127.19.183.253:40421): Couldn't send request to peer b399be0c91ba480a873d964b624c404e. Status: Network error: Client connection negotiation failed: client connection to 127.19.183.253:40421: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 12:17:22.466249 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 29752
2026-08-09T12:17:22Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (10330 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:17:22 dist-test-slave-39l8 krb5kdc[31025](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:17:22 dist-test-slave-39l8 krb5kdc[31025](info): set up 2 sockets
Aug 09 12:17:22 dist-test-slave-39l8 krb5kdc[31025](info): commencing operation
krb5kdc: starting...
W20260809 12:17:23.786957 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.263s user 0.000s sys 0.035s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:17:23 dist-test-slave-39l8 krb5kdc[31025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:17:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:17:23Z Disabled control of system clock
WARNING: no policy specified for oryx/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.254.keytab.
Entry for principal oryx/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.254.keytab.
I20260809 12:17:24.052038 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:43903
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.254.keytab
--principal=oryx/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:34093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:43903
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:17:24.389595 31045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:24.389878 31045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:24.389966 31045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:24.397337 31045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:17:24.397452 31045 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 12:17:24.397495 31045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:24.397583 31045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:24.397670 31045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:17:24.397784 31045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:17:24.407554 31045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:34093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:43903
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.254.keytab
--principal=oryx/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:43903
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:24.408762 31045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:24.410087 31045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:24.421972 31059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:24.421998 31054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:24.421944 31056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:24.422921 31045 server_base.cc:1034] running on GCE node
I20260809 12:17:24.423663 31045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:24.425050 31045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:24.426288 31045 hybrid_clock.cc:648] HybridClock initialized: now 1786277844426221 us; error 42 us; skew 500 ppm
I20260809 12:17:24.426601 31045 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:24 dist-test-slave-39l8 krb5kdc[31025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277844, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:24.429785 31045 init.cc:377] Logged in from keytab as oryx/127.19.183.254@KRBTEST.COM (short username oryx)
I20260809 12:17:24.431183 31045 webserver.cc:460] Webserver started at http://127.19.183.254:39973/ using document root <none> and password file <none>
I20260809 12:17:24.431808 31045 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:24.431895 31045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:24.432171 31045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:24.435099 31045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "ddb76241bc9a4ecdbb72c7b7bbe9a112"
format_stamp: "Formatted at 2026-08-09 12:17:24 on dist-test-slave-39l8"
server_key: "6fa9ff18b4dd2a106a348ee54d3da5b4"
server_key_iv: "c30b33024310bc68efa207dbb97937c9"
server_key_version: "encryptionkey@0"
I20260809 12:17:24.435853 31045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "ddb76241bc9a4ecdbb72c7b7bbe9a112"
format_stamp: "Formatted at 2026-08-09 12:17:24 on dist-test-slave-39l8"
server_key: "6fa9ff18b4dd2a106a348ee54d3da5b4"
server_key_iv: "c30b33024310bc68efa207dbb97937c9"
server_key_version: "encryptionkey@0"
I20260809 12:17:24.440755 31045 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:17:24.444154 31070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:24.445529 31045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:17:24.445708 31045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "ddb76241bc9a4ecdbb72c7b7bbe9a112"
format_stamp: "Formatted at 2026-08-09 12:17:24 on dist-test-slave-39l8"
server_key: "6fa9ff18b4dd2a106a348ee54d3da5b4"
server_key_iv: "c30b33024310bc68efa207dbb97937c9"
server_key_version: "encryptionkey@0"
I20260809 12:17:24.445945 31045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:24.460670 31045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:24.465153 31045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:24.465467 31045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:24.487321 31045 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:43903
I20260809 12:17:24.487346 31153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:43903 every 8 connection(s)
I20260809 12:17:24.488749 31045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:17:24.492420 31154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:24.497339 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 31045
I20260809 12:17:24.497555 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:17:24.498381 20191 external_mini_cluster.cc:1428] Setting key 4583d5329ef7003a401ea4cf67178f9e
I20260809 12:17:24.504282 31154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112: Bootstrap starting.
I20260809 12:17:24.507849 31154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:24.509037 31154 log.cc:824] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112: Log is configured to *not* fsync() on all Append() calls
I20260809 12:17:24.512436 31154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112: No bootstrap required, opened a new log
Aug 09 12:17:24 dist-test-slave-39l8 krb5kdc[31025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277843, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM
I20260809 12:17:24.520156 31154 raft_consensus.cc:348] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb76241bc9a4ecdbb72c7b7bbe9a112" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 43903 } }
I20260809 12:17:24.520663 31154 raft_consensus.cc:374] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:24.520809 31154 raft_consensus.cc:729] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddb76241bc9a4ecdbb72c7b7bbe9a112, State: Initialized, Role: FOLLOWER
I20260809 12:17:24.521517 31154 consensus_queue.cc:260] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb76241bc9a4ecdbb72c7b7bbe9a112" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 43903 } }
I20260809 12:17:24.521797 31154 raft_consensus.cc:388] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:17:24.521960 31158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:24.500410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53138 (local address 127.19.183.254:43903)
0809 12:17:24.501231 (+ 821us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:24.501242 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:24.501287 (+ 45us) server_negotiation.cc:407] Connection header received
0809 12:17:24.503229 (+ 1942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:24.503253 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:24.503618 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:24.503921 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:24.505730 (+ 1809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:24.506998 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:24.508678 (+ 1680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:24.509374 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:24.509681 (+ 307us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:24.514212 (+ 4531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:24.514219 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:24.514235 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:24.514268 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:24.517281 (+ 3013us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:24.517940 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:24.517947 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:24.517953 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:24.518109 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:24.518504 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:24.518509 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:24.518514 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:24.519304 (+ 790us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:24.519605 (+ 301us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:24.520087 (+ 482us) server_negotiation.cc:299] Negotiation successful
0809 12:17:24.520524 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":403,"threads_started":1}
I20260809 12:17:24.522009 31154 raft_consensus.cc:482] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:17:24.522248 31154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:24.524996 31154 raft_consensus.cc:504] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb76241bc9a4ecdbb72c7b7bbe9a112" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 43903 } }
I20260809 12:17:24.525522 31154 leader_election.cc:302] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ddb76241bc9a4ecdbb72c7b7bbe9a112; no voters:
I20260809 12:17:24.526114 31154 leader_election.cc:288] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:17:24.526249 31162 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:17:24.526969 31162 raft_consensus.cc:686] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112 [term 1 LEADER]: Becoming Leader. State: Replica: ddb76241bc9a4ecdbb72c7b7bbe9a112, State: Running, Role: LEADER
I20260809 12:17:24.527531 31162 consensus_queue.cc:237] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb76241bc9a4ecdbb72c7b7bbe9a112" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 43903 } }
I20260809 12:17:24.528206 31154 sys_catalog.cc:564] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:17:24.530050 31163 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ddb76241bc9a4ecdbb72c7b7bbe9a112" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb76241bc9a4ecdbb72c7b7bbe9a112" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 43903 } } }
I20260809 12:17:24.531252 31163 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112 [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:24.532061 31164 sys_catalog.cc:455] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ddb76241bc9a4ecdbb72c7b7bbe9a112. Latest consensus state: current_term: 1 leader_uuid: "ddb76241bc9a4ecdbb72c7b7bbe9a112" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ddb76241bc9a4ecdbb72c7b7bbe9a112" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 43903 } } }
I20260809 12:17:24.532485 31164 sys_catalog.cc:458] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112 [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:24.536655 31174 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:17:24.542991 31174 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:17:24.554129 31174 catalog_manager.cc:1347] Generated new cluster ID: 186a3410a95946f6a352894cb9249ecf
I20260809 12:17:24.554257 31174 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:17:24.562409 31174 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:17:24.563661 31174 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:17:24.576896 31174 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112: Generated new TSK 0
I20260809 12:17:24.577634 31174 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.19.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.193.keytab.
Entry for principal oryx/127.19.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.193.keytab.
I20260809 12:17:24.658481 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:43903
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.193.keytab
--principal=oryx/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:34093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:17:25.041916 31191 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:25.042276 31191 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:25.042466 31191 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:25.050007 31191 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:25.050154 31191 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:25.050292 31191 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:17:25.050436 31191 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:17:25.060295 31191 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:34093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.193.keytab
--principal=oryx/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.183.254:43903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:25.061610 31191 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:25.063091 31191 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:25.076462 31199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:25.076550 31197 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:25.076571 31201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:25.077692 31191 server_base.cc:1034] running on GCE node
I20260809 12:17:25.078248 31191 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:25.079237 31191 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:25.080446 31191 hybrid_clock.cc:648] HybridClock initialized: now 1786277845080369 us; error 52 us; skew 500 ppm
I20260809 12:17:25.080705 31191 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:25 dist-test-slave-39l8 krb5kdc[31025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277845, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:25.083927 31191 init.cc:377] Logged in from keytab as oryx/127.19.183.193@KRBTEST.COM (short username oryx)
I20260809 12:17:25.085443 31191 webserver.cc:460] Webserver started at http://127.19.183.193:38767/ using document root <none> and password file <none>
I20260809 12:17:25.086191 31191 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:25.086311 31191 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:25.086659 31191 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:25.089517 31191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "6f753bec541d42d9bcf7da08421aae48"
format_stamp: "Formatted at 2026-08-09 12:17:25 on dist-test-slave-39l8"
server_key: "cde69c704332a4a8c40f6f5ae1f8030f"
server_key_iv: "8c49caefee5dbd314327fc2d38d60de6"
server_key_version: "encryptionkey@0"
I20260809 12:17:25.090335 31191 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "6f753bec541d42d9bcf7da08421aae48"
format_stamp: "Formatted at 2026-08-09 12:17:25 on dist-test-slave-39l8"
server_key: "cde69c704332a4a8c40f6f5ae1f8030f"
server_key_iv: "8c49caefee5dbd314327fc2d38d60de6"
server_key_version: "encryptionkey@0"
I20260809 12:17:25.095412 31191 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260809 12:17:25.098874 31212 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:25.100301 31191 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:17:25.100473 31191 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "6f753bec541d42d9bcf7da08421aae48"
format_stamp: "Formatted at 2026-08-09 12:17:25 on dist-test-slave-39l8"
server_key: "cde69c704332a4a8c40f6f5ae1f8030f"
server_key_iv: "8c49caefee5dbd314327fc2d38d60de6"
server_key_version: "encryptionkey@0"
I20260809 12:17:25.100683 31191 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:25.122017 31191 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:25.126253 31191 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:25.126533 31191 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:25.129827 31191 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:25.129952 31191 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:25.130146 31191 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:25.130231 31191 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:17:25 dist-test-slave-39l8 krb5kdc[31025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277845, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM
I20260809 12:17:25.155827 31226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:25.130064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:43855 (local address 127.19.183.254:43903)
0809 12:17:25.130631 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:25.130640 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:25.131977 (+ 1337us) server_negotiation.cc:407] Connection header received
0809 12:17:25.134958 (+ 2981us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:25.134963 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:25.135080 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:25.135216 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:25.137859 (+ 2643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:25.138849 (+ 990us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:25.140657 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:25.141116 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:25.141260 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:25.146172 (+ 4912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:25.146177 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:25.146181 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:25.146233 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:25.149993 (+ 3760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:25.151262 (+ 1269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:25.151269 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:25.151274 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:25.151441 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:25.152119 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:25.152127 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:25.152132 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:25.152537 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:25.154882 (+ 2345us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:25.155084 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 12:17:25.155360 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":265,"threads_started":1}
I20260809 12:17:25.156191 31230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:25.130789 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:43903 (local address 127.19.183.193:43855)
0809 12:17:25.131765 (+ 976us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:25.131832 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:25.134633 (+ 2801us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:25.135532 (+ 899us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:25.135542 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:25.136429 (+ 887us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:25.137624 (+ 1195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:25.137657 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:25.139005 (+ 1348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:25.139011 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:25.140507 (+ 1496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:25.140521 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:25.141368 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:25.141377 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:25.141608 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:25.142330 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:25.142384 (+ 54us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:25.145929 (+ 3545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:25.150202 (+ 4273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:25.150211 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:25.150230 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:25.151045 (+ 815us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:25.151594 (+ 549us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:25.151604 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:25.151610 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:25.151916 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:25.152721 (+ 805us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:25.152731 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:25.153321 (+ 590us) client_negotiation.cc:769] Sending connection context
0809 12:17:25.153677 (+ 356us) client_negotiation.cc:240] Negotiation successful
0809 12:17:25.154344 (+ 667us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":634,"thread_start_us":332,"threads_started":1}
I20260809 12:17:25.174619 31191 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:42711
I20260809 12:17:25.174625 31408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:42711 every 8 connection(s)
I20260809 12:17:25.176005 31191 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:17:25.178208 31409 heartbeater.cc:347] Connected to a master server at 127.19.183.254:43903
I20260809 12:17:25.178454 31409 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:25.179293 31409 heartbeater.cc:511] Master 127.19.183.254:43903 requested a full tablet report, sending...
I20260809 12:17:25.180747 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 31191
I20260809 12:17:25.180960 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:17:25.181363 31226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:25.165066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:51855 (local address 127.19.183.254:43903)
0809 12:17:25.165301 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:25.165310 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:25.166256 (+ 946us) server_negotiation.cc:407] Connection header received
0809 12:17:25.166394 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:25.166403 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:25.166494 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:25.166606 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:25.168294 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:25.169582 (+ 1288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:25.171826 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:25.172402 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:25.172560 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:25.175083 (+ 2523us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:25.175088 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:25.175092 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:25.175121 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:25.178276 (+ 3155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:25.179134 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:25.179143 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:25.179148 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:25.179309 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:25.179710 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:25.179714 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:25.179718 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:25.180069 (+ 351us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:25.180218 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:25.180737 (+ 519us) server_negotiation.cc:299] Negotiation successful
0809 12:17:25.180951 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260809 12:17:25.181396 31371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:25.165105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:43903 (local address 127.19.183.193:51855)
0809 12:17:25.166137 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:25.166155 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:25.166278 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:25.166665 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:25.166673 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:25.167485 (+ 812us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:25.168107 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:25.168131 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:25.169798 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:25.169803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:25.171583 (+ 1780us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:25.171599 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:25.172642 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:25.172649 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:25.172813 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:25.173547 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:25.173578 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:25.174897 (+ 1319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:25.178452 (+ 3555us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:25.178456 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:25.178458 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:25.178969 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:25.179434 (+ 465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:25.179438 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:25.179441 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:25.179581 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:25.180209 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:25.180213 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:25.180425 (+ 212us) client_negotiation.cc:769] Sending connection context
0809 12:17:25.180623 (+ 198us) client_negotiation.cc:240] Negotiation successful
0809 12:17:25.180943 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":879,"thread_start_us":731,"threads_started":1}
I20260809 12:17:25.181793 20191 external_mini_cluster.cc:1428] Setting key e7ccb65a69188e82ee254570cbd22925
I20260809 12:17:25.181816 31091 ts_manager.cc:194] Registered new tserver with Master: 6f753bec541d42d9bcf7da08421aae48 (127.19.183.193:42711)
I20260809 12:17:25.183495 31091 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:43855
WARNING: no policy specified for oryx/127.19.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.194.keytab.
Entry for principal oryx/127.19.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.194.keytab.
I20260809 12:17:25.239300 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:43903
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.194.keytab
--principal=oryx/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:34093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:17:25.575490 31414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:25.575796 31414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:25.575969 31414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:25.583220 31414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:25.583359 31414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:25.583503 31414 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:17:25.583635 31414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:17:25.593034 31414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:34093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.194.keytab
--principal=oryx/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.183.254:43903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:25.594229 31414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:25.595589 31414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:25.608669 31422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:25.608743 31424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:25.608729 31421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:25.609301 31414 server_base.cc:1034] running on GCE node
I20260809 12:17:25.609933 31414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:25.610929 31414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:25.612193 31414 hybrid_clock.cc:648] HybridClock initialized: now 1786277845612108 us; error 48 us; skew 500 ppm
I20260809 12:17:25.612484 31414 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:25 dist-test-slave-39l8 krb5kdc[31025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277845, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:25.620289 31414 init.cc:377] Logged in from keytab as oryx/127.19.183.194@KRBTEST.COM (short username oryx)
I20260809 12:17:25.621686 31414 webserver.cc:460] Webserver started at http://127.19.183.194:34013/ using document root <none> and password file <none>
I20260809 12:17:25.622367 31414 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:25.622504 31414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:25.622943 31414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:25.625799 31414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "918856a3d1054b24b5c3410deca11388"
format_stamp: "Formatted at 2026-08-09 12:17:25 on dist-test-slave-39l8"
server_key: "53b64323ab8fcc55fba74ab3443e0575"
server_key_iv: "8728d20ce32c163885777bddca2007d6"
server_key_version: "encryptionkey@0"
I20260809 12:17:25.626693 31414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "918856a3d1054b24b5c3410deca11388"
format_stamp: "Formatted at 2026-08-09 12:17:25 on dist-test-slave-39l8"
server_key: "53b64323ab8fcc55fba74ab3443e0575"
server_key_iv: "8728d20ce32c163885777bddca2007d6"
server_key_version: "encryptionkey@0"
I20260809 12:17:25.631821 31414 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 12:17:25.635306 31433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:25.636579 31414 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:17:25.636780 31414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "918856a3d1054b24b5c3410deca11388"
format_stamp: "Formatted at 2026-08-09 12:17:25 on dist-test-slave-39l8"
server_key: "53b64323ab8fcc55fba74ab3443e0575"
server_key_iv: "8728d20ce32c163885777bddca2007d6"
server_key_version: "encryptionkey@0"
I20260809 12:17:25.637002 31414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:25.654511 31414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:25.658392 31414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:25.658757 31414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:25.661696 31414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:25.661820 31414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 12:17:25.661981 31414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:25.662065 31414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:17:25 dist-test-slave-39l8 krb5kdc[31025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277845, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM
I20260809 12:17:25.686877 31226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:25.661855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:40381 (local address 127.19.183.254:43903)
0809 12:17:25.662066 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:25.662075 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:25.663518 (+ 1443us) server_negotiation.cc:407] Connection header received
0809 12:17:25.665688 (+ 2170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:25.665694 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:25.665791 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:25.665905 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:25.667950 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:25.668936 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:25.672448 (+ 3512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:25.672896 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:25.673061 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:25.678610 (+ 5549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:25.678618 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:25.678623 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:25.678669 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:25.682821 (+ 4152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:25.683761 (+ 940us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:25.683768 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:25.683773 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:25.683926 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:25.684403 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:25.684408 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:25.684412 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:25.684854 (+ 442us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:25.685116 (+ 262us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:25.685999 (+ 883us) server_negotiation.cc:299] Negotiation successful
0809 12:17:25.686314 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 12:17:25.687600 31448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:25.662424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:43903 (local address 127.19.183.194:40381)
0809 12:17:25.663345 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:25.663395 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:25.665392 (+ 1997us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:25.666144 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:25.666152 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:25.666898 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:25.667693 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:25.667716 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:25.669154 (+ 1438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:25.669162 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:25.672290 (+ 3128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:25.672307 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:25.673178 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:25.673188 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:25.673605 (+ 417us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:25.674494 (+ 889us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:25.674518 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:25.678296 (+ 3778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:25.682997 (+ 4701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:25.683009 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:25.683027 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:25.683611 (+ 584us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:25.684047 (+ 436us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:25.684052 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:25.684056 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:25.684273 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:25.685046 (+ 773us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:25.685055 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:25.685429 (+ 374us) client_negotiation.cc:769] Sending connection context
0809 12:17:25.685779 (+ 350us) client_negotiation.cc:240] Negotiation successful
0809 12:17:25.686262 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":638,"thread_start_us":417,"threads_started":1}
I20260809 12:17:25.710255 31547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:25.693382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:43903 (local address 127.19.183.194:42859)
0809 12:17:25.694167 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:25.694182 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:25.694338 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:25.694780 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:25.694788 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:25.695534 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:25.696023 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:25.696040 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:25.697064 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:25.697072 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:25.699213 (+ 2141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:25.699235 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:25.700395 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:25.700400 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:25.700568 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:25.701297 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:25.701313 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:25.702548 (+ 1235us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:25.707255 (+ 4707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:25.707266 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:25.707270 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:25.707844 (+ 574us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:25.708359 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:25.708371 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:25.708378 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:25.708590 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:25.709409 (+ 819us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:25.709413 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:25.709558 (+ 145us) client_negotiation.cc:769] Sending connection context
0809 12:17:25.709692 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 12:17:25.709931 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":663,"thread_start_us":513,"threads_started":1}
I20260809 12:17:25.710425 31226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:25.693283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:42859 (local address 127.19.183.254:43903)
0809 12:17:25.693466 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:25.693473 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:25.694269 (+ 796us) server_negotiation.cc:407] Connection header received
0809 12:17:25.694468 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:25.694474 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:25.694575 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:25.694692 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:25.696206 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:25.696921 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:25.699445 (+ 2524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:25.699908 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:25.700105 (+ 197us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:25.702844 (+ 2739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:25.702854 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:25.702862 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:25.702913 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:25.707100 (+ 4187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:25.707985 (+ 885us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:25.707993 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:25.707996 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:25.708198 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:25.708761 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:25.708770 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:25.708775 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:25.709262 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:25.709435 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:25.709826 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 12:17:25.710056 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 12:17:25.720675 31414 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:43999
I20260809 12:17:25.720709 31614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:43999 every 8 connection(s)
I20260809 12:17:25.721923 31414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:17:25.724045 31615 heartbeater.cc:347] Connected to a master server at 127.19.183.254:43903
I20260809 12:17:25.724239 31615 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:25.724692 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 31414
I20260809 12:17:25.724870 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:17:25.724886 31615 heartbeater.cc:511] Master 127.19.183.254:43903 requested a full tablet report, sending...
I20260809 12:17:25.725335 20191 external_mini_cluster.cc:1428] Setting key 799c690981a5e67fd18d60996e142f5f
I20260809 12:17:25.726418 31090 ts_manager.cc:194] Registered new tserver with Master: 918856a3d1054b24b5c3410deca11388 (127.19.183.194:43999)
I20260809 12:17:25.727504 31090 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:40381
WARNING: no policy specified for oryx/127.19.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.195.keytab.
Entry for principal oryx/127.19.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.195.keytab.
I20260809 12:17:25.781744 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:43903
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.195.keytab
--principal=oryx/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:34093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:17:26.116544 31619 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:26.116815 31619 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:26.116931 31619 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:26.124037 31619 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:26.124173 31619 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:26.124338 31619 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 12:17:26.124464 31619 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:17:26.134137 31619 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:34093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.195.keytab
--principal=oryx/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.19.183.254:43903
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:26.135360 31619 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:26.136700 31619 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:26.149782 31626 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:26.149834 31628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:26.150069 31630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:26.150725 31619 server_base.cc:1034] running on GCE node
I20260809 12:17:26.151386 31619 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:26.152302 31619 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:26.153548 31619 hybrid_clock.cc:648] HybridClock initialized: now 1786277846153481 us; error 44 us; skew 500 ppm
I20260809 12:17:26.153844 31619 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:26 dist-test-slave-39l8 krb5kdc[31025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277846, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:26.156949 31619 init.cc:377] Logged in from keytab as oryx/127.19.183.195@KRBTEST.COM (short username oryx)
I20260809 12:17:26.158263 31619 webserver.cc:460] Webserver started at http://127.19.183.195:35363/ using document root <none> and password file <none>
I20260809 12:17:26.159031 31619 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:26.159180 31619 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:26.159510 31619 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:26.162499 31619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "5f34776fb824418b99b2d9889300b449"
format_stamp: "Formatted at 2026-08-09 12:17:26 on dist-test-slave-39l8"
server_key: "edaf928758274cdc48bbf92082ea3c6d"
server_key_iv: "6a48a4493a74fef901ae864ce398b778"
server_key_version: "encryptionkey@0"
I20260809 12:17:26.163460 31619 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "5f34776fb824418b99b2d9889300b449"
format_stamp: "Formatted at 2026-08-09 12:17:26 on dist-test-slave-39l8"
server_key: "edaf928758274cdc48bbf92082ea3c6d"
server_key_iv: "6a48a4493a74fef901ae864ce398b778"
server_key_version: "encryptionkey@0"
I20260809 12:17:26.168608 31619 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:17:26.172006 31640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:26.173260 31619 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:17:26.173421 31619 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "5f34776fb824418b99b2d9889300b449"
format_stamp: "Formatted at 2026-08-09 12:17:26 on dist-test-slave-39l8"
server_key: "edaf928758274cdc48bbf92082ea3c6d"
server_key_iv: "6a48a4493a74fef901ae864ce398b778"
server_key_version: "encryptionkey@0"
I20260809 12:17:26.173643 31619 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:26.186641 31409 heartbeater.cc:503] Master 127.19.183.254:43903 was elected leader, sending a full tablet report...
I20260809 12:17:26.188664 31619 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:26.192535 31619 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:26.192806 31619 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:26.195529 31619 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:26.195653 31619 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:26.195809 31619 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:26.195886 31619 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:17:26 dist-test-slave-39l8 krb5kdc[31025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277846, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM
I20260809 12:17:26.220191 31226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:26.195949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:60305 (local address 127.19.183.254:43903)
0809 12:17:26.196161 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:26.196171 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:26.197627 (+ 1456us) server_negotiation.cc:407] Connection header received
0809 12:17:26.200604 (+ 2977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:26.200612 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:26.200703 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:26.200822 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:26.203293 (+ 2471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.204771 (+ 1478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.207326 (+ 2555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.207963 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.208105 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:26.212410 (+ 4305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:26.212418 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:26.212422 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:26.212470 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:26.216277 (+ 3807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:26.217291 (+ 1014us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:26.217300 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:26.217306 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:26.217516 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:26.217987 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:26.217993 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:26.217997 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:26.218317 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:26.218481 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:26.219594 (+ 1113us) server_negotiation.cc:299] Negotiation successful
0809 12:17:26.219830 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 12:17:26.221590 31651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:26.196466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:43903 (local address 127.19.183.195:60305)
0809 12:17:26.197383 (+ 917us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:26.197438 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:26.200265 (+ 2827us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:26.201256 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:26.201269 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:26.202111 (+ 842us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:26.202993 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:26.203016 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.204948 (+ 1932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.204952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:26.207182 (+ 2230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:26.207195 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.208211 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.208219 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:26.208429 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:26.209057 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:26.209078 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:26.212122 (+ 3044us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:26.216458 (+ 4336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:26.216473 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:26.216487 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:26.217120 (+ 633us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:26.217648 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:26.217651 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:26.217653 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:26.217848 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:26.218461 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:26.218468 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:26.218969 (+ 501us) client_negotiation.cc:769] Sending connection context
0809 12:17:26.219443 (+ 474us) client_negotiation.cc:240] Negotiation successful
0809 12:17:26.220010 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":619,"thread_start_us":365,"threads_started":1}
I20260809 12:17:26.243033 31226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:26.228158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:49595 (local address 127.19.183.254:43903)
0809 12:17:26.228295 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:26.228299 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:26.228923 (+ 624us) server_negotiation.cc:407] Connection header received
0809 12:17:26.229205 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:26.229210 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:26.229331 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:26.229433 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:26.230342 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.231308 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.233361 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.233953 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.234088 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:26.236730 (+ 2642us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:26.236736 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:26.236742 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:26.236803 (+ 61us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:26.239972 (+ 3169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:26.240940 (+ 968us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:26.240947 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:26.240950 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:26.241081 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:26.241456 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:26.241461 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:26.241465 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:26.241833 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:26.241989 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:26.242408 (+ 419us) server_negotiation.cc:299] Negotiation successful
0809 12:17:26.242635 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 12:17:26.243189 31749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:26.228202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:43903 (local address 127.19.183.195:49595)
0809 12:17:26.228809 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:26.228827 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:26.228998 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:26.229454 (+ 456us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:26.229458 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:26.229894 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:26.230202 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:26.230210 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.231435 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.231441 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:26.233136 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:26.233163 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.234337 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.234342 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:26.234463 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:26.235308 (+ 845us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:26.235337 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:26.236433 (+ 1096us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:26.240131 (+ 3698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:26.240136 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:26.240140 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:26.240806 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:26.241205 (+ 399us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:26.241210 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:26.241214 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:26.241343 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:26.241959 (+ 616us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:26.241965 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:26.242156 (+ 191us) client_negotiation.cc:769] Sending connection context
0809 12:17:26.242301 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 12:17:26.242616 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":492,"thread_start_us":365,"threads_started":1}
I20260809 12:17:26.251890 31619 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:35519
I20260809 12:17:26.251928 31829 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:35519 every 8 connection(s)
I20260809 12:17:26.253108 31619 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:17:26.255129 31830 heartbeater.cc:347] Connected to a master server at 127.19.183.254:43903
I20260809 12:17:26.255362 31830 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:26.255968 31830 heartbeater.cc:511] Master 127.19.183.254:43903 requested a full tablet report, sending...
I20260809 12:17:26.257339 31090 ts_manager.cc:194] Registered new tserver with Master: 5f34776fb824418b99b2d9889300b449 (127.19.183.195:35519)
I20260809 12:17:26.257493 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 31619
I20260809 12:17:26.257692 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:17:26.258116 20191 external_mini_cluster.cc:1428] Setting key c785b8ad720d66f66291d30aa8c01647
I20260809 12:17:26.258280 31090 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:60305
I20260809 12:17:26.261055 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 12:17:26 dist-test-slave-39l8 krb5kdc[31025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277846, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 12:17:26 dist-test-slave-39l8 krb5kdc[31025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:26 dist-test-slave-39l8 krb5kdc[31025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 12:17:26.323047 31226 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:26.312436 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53154 (local address 127.19.183.254:43903)
0809 12:17:26.312701 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:26.312708 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:26.312936 (+ 228us) server_negotiation.cc:407] Connection header received
0809 12:17:26.313091 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:26.313095 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:26.313163 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:26.313248 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:26.314477 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.315270 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.316509 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.316907 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.317003 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:26.322639 (+ 5636us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53154: BlockingRecv error: recv got EOF from 127.0.0.1:53154 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 09 12:17:26 dist-test-slave-39l8 krb5kdc[31025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277846, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM
I20260809 12:17:26.340020 31226 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:26.326299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53160 (local address 127.19.183.254:43903)
0809 12:17:26.326520 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:26.326525 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:26.326818 (+ 293us) server_negotiation.cc:407] Connection header received
0809 12:17:26.327009 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:26.327020 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:26.327092 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:26.327197 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:26.328448 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.329161 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.330381 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.330873 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.331034 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:26.334763 (+ 3729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:26.334768 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:26.334772 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:26.334805 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:26.337385 (+ 2580us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:26.338013 (+ 628us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:26.338023 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:26.338025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:26.338141 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:26.338513 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:26.338518 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:26.338521 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:26.338851 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:26.339218 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:26.339480 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 12:17:26.339735 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 12:17:26.343986 31090 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53160:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:17:26.350853 31090 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:17:26.385280 31863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:26.362703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:35519 (local address 127.0.0.1:50392)
0809 12:17:26.375872 (+ 13169us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:26.375910 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:26.376803 (+ 893us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:26.377663 (+ 860us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:26.377666 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:26.377688 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:17:26.378271 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:26.378287 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.380338 (+ 2051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.380344 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:26.382045 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:26.382059 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.384041 (+ 1982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.384047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:26.384265 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:26.384321 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 12:17:26.384468 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 12:17:26.384663 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12883,"mutex_wait_us":7,"thread_start_us":12952,"threads_started":1}
I20260809 12:17:26.385418 31858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:26.363216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34530 (local address 127.19.183.194:43999)
0809 12:17:26.374493 (+ 11277us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:26.374506 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:26.376627 (+ 2121us) server_negotiation.cc:407] Connection header received
0809 12:17:26.376821 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:26.376830 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:26.377058 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:26.377277 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:17:26.378285 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.379845 (+ 1560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.382891 (+ 3046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.384288 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.384483 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:26.384649 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:26.384922 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 12:17:26.385106 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11041,"thread_start_us":11272,"threads_started":1}
I20260809 12:17:26.386077 31856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:26.363079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50392 (local address 127.19.183.195:35519)
0809 12:17:26.375430 (+ 12351us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:26.375442 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:26.376017 (+ 575us) server_negotiation.cc:407] Connection header received
0809 12:17:26.376992 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:26.377000 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:26.377256 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:26.377392 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:17:26.378497 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.380167 (+ 1670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.382224 (+ 2057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.383751 (+ 1527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.384075 (+ 324us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:26.384293 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:26.385563 (+ 1270us) server_negotiation.cc:299] Negotiation successful
0809 12:17:26.385731 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12123,"thread_start_us":11836,"threads_started":1}
I20260809 12:17:26.385852 31855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:26.363492 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:42711 (local address 127.0.0.1:43402)
0809 12:17:26.375968 (+ 12476us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:26.375986 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:26.376200 (+ 214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:26.377216 (+ 1016us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:26.377223 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:26.377258 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:17:26.377773 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:26.377790 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.379547 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.379554 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:26.382728 (+ 3174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:26.382743 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.384882 (+ 2139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.384889 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:26.385046 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:26.385091 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 12:17:26.385306 (+ 215us) client_negotiation.cc:240] Negotiation successful
0809 12:17:26.385465 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12358,"mutex_wait_us":203,"thread_start_us":11953,"threads_started":1}
I20260809 12:17:26.386466 31859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:26.364253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43402 (local address 127.19.183.193:42711)
0809 12:17:26.374437 (+ 10184us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:26.374450 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:26.376126 (+ 1676us) server_negotiation.cc:407] Connection header received
0809 12:17:26.376458 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:26.376464 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:26.376743 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:26.376946 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:17:26.378040 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.379342 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.382920 (+ 3578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.384368 (+ 1448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.384529 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:26.384708 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:26.385987 (+ 1279us) server_negotiation.cc:299] Negotiation successful
0809 12:17:26.386177 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9989,"thread_start_us":9843,"threads_started":1}
I20260809 12:17:26.385908 31861 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:26.363630 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:43999 (local address 127.0.0.1:34530)
0809 12:17:26.376491 (+ 12861us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:26.376507 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:26.376647 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:26.377509 (+ 862us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:26.377516 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:26.377543 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:17:26.378124 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:26.378136 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.379981 (+ 1845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.379986 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:26.382730 (+ 2744us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:26.382743 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.384507 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.384514 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:26.384691 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:26.384724 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 12:17:26.384871 (+ 147us) client_negotiation.cc:240] Negotiation successful
0809 12:17:26.385006 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12746,"mutex_wait_us":189,"thread_start_us":12582,"threads_started":1}
I20260809 12:17:26.390064 31717 tablet_service.cc:1467] Processing CreateTablet for tablet 747d79a25a8e45abb83ef0a493ed475d (DEFAULT_TABLE table=test-table [id=360ce4ef8e82417ca02785f2ef7a3a89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:17:26.390053 31290 tablet_service.cc:1467] Processing CreateTablet for tablet 747d79a25a8e45abb83ef0a493ed475d (DEFAULT_TABLE table=test-table [id=360ce4ef8e82417ca02785f2ef7a3a89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:17:26.391397 31717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 747d79a25a8e45abb83ef0a493ed475d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:26.391532 31290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 747d79a25a8e45abb83ef0a493ed475d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:26.393918 31514 tablet_service.cc:1467] Processing CreateTablet for tablet 747d79a25a8e45abb83ef0a493ed475d (DEFAULT_TABLE table=test-table [id=360ce4ef8e82417ca02785f2ef7a3a89]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:17:26.395051 31514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 747d79a25a8e45abb83ef0a493ed475d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:26.402870 31864 tablet_bootstrap.cc:492] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48: Bootstrap starting.
I20260809 12:17:26.407356 31864 tablet_bootstrap.cc:654] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:26.408560 31864 log.cc:824] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48: Log is configured to *not* fsync() on all Append() calls
I20260809 12:17:26.408917 31865 tablet_bootstrap.cc:492] T 747d79a25a8e45abb83ef0a493ed475d P 918856a3d1054b24b5c3410deca11388: Bootstrap starting.
I20260809 12:17:26.412544 31867 tablet_bootstrap.cc:492] T 747d79a25a8e45abb83ef0a493ed475d P 5f34776fb824418b99b2d9889300b449: Bootstrap starting.
I20260809 12:17:26.413740 31865 tablet_bootstrap.cc:654] T 747d79a25a8e45abb83ef0a493ed475d P 918856a3d1054b24b5c3410deca11388: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:26.415196 31865 log.cc:824] T 747d79a25a8e45abb83ef0a493ed475d P 918856a3d1054b24b5c3410deca11388: Log is configured to *not* fsync() on all Append() calls
I20260809 12:17:26.416180 31867 tablet_bootstrap.cc:654] T 747d79a25a8e45abb83ef0a493ed475d P 5f34776fb824418b99b2d9889300b449: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:26.416391 31864 tablet_bootstrap.cc:492] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48: No bootstrap required, opened a new log
I20260809 12:17:26.416710 31864 ts_tablet_manager.cc:1397] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48: Time spent bootstrapping tablet: real 0.014s user 0.010s sys 0.000s
I20260809 12:17:26.417414 31867 log.cc:824] T 747d79a25a8e45abb83ef0a493ed475d P 5f34776fb824418b99b2d9889300b449: Log is configured to *not* fsync() on all Append() calls
I20260809 12:17:26.417920 31865 tablet_bootstrap.cc:492] T 747d79a25a8e45abb83ef0a493ed475d P 918856a3d1054b24b5c3410deca11388: No bootstrap required, opened a new log
I20260809 12:17:26.418215 31865 ts_tablet_manager.cc:1397] T 747d79a25a8e45abb83ef0a493ed475d P 918856a3d1054b24b5c3410deca11388: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.003s
I20260809 12:17:26.420689 31867 tablet_bootstrap.cc:492] T 747d79a25a8e45abb83ef0a493ed475d P 5f34776fb824418b99b2d9889300b449: No bootstrap required, opened a new log
I20260809 12:17:26.420964 31867 ts_tablet_manager.cc:1397] T 747d79a25a8e45abb83ef0a493ed475d P 5f34776fb824418b99b2d9889300b449: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.000s
I20260809 12:17:26.426596 31865 raft_consensus.cc:348] T 747d79a25a8e45abb83ef0a493ed475d P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } }
I20260809 12:17:26.427021 31865 raft_consensus.cc:374] T 747d79a25a8e45abb83ef0a493ed475d P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:26.427134 31865 raft_consensus.cc:729] T 747d79a25a8e45abb83ef0a493ed475d P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 918856a3d1054b24b5c3410deca11388, State: Initialized, Role: FOLLOWER
I20260809 12:17:26.427454 31864 raft_consensus.cc:348] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } }
I20260809 12:17:26.427873 31864 raft_consensus.cc:374] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:26.427968 31864 raft_consensus.cc:729] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f753bec541d42d9bcf7da08421aae48, State: Initialized, Role: FOLLOWER
I20260809 12:17:26.427760 31865 consensus_queue.cc:260] T 747d79a25a8e45abb83ef0a493ed475d P 918856a3d1054b24b5c3410deca11388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } }
I20260809 12:17:26.428570 31864 consensus_queue.cc:260] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } }
I20260809 12:17:26.429245 31615 heartbeater.cc:503] Master 127.19.183.254:43903 was elected leader, sending a full tablet report...
I20260809 12:17:26.429960 31865 ts_tablet_manager.cc:1428] T 747d79a25a8e45abb83ef0a493ed475d P 918856a3d1054b24b5c3410deca11388: Time spent starting tablet: real 0.012s user 0.012s sys 0.001s
I20260809 12:17:26.431856 31864 ts_tablet_manager.cc:1428] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48: Time spent starting tablet: real 0.015s user 0.009s sys 0.007s
I20260809 12:17:26.432514 31867 raft_consensus.cc:348] T 747d79a25a8e45abb83ef0a493ed475d P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } }
I20260809 12:17:26.432932 31867 raft_consensus.cc:374] T 747d79a25a8e45abb83ef0a493ed475d P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:26.433053 31867 raft_consensus.cc:729] T 747d79a25a8e45abb83ef0a493ed475d P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f34776fb824418b99b2d9889300b449, State: Initialized, Role: FOLLOWER
I20260809 12:17:26.433912 31867 consensus_queue.cc:260] T 747d79a25a8e45abb83ef0a493ed475d P 5f34776fb824418b99b2d9889300b449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } }
I20260809 12:17:26.435071 31830 heartbeater.cc:503] Master 127.19.183.254:43903 was elected leader, sending a full tablet report...
I20260809 12:17:26.435750 31867 ts_tablet_manager.cc:1428] T 747d79a25a8e45abb83ef0a493ed475d P 5f34776fb824418b99b2d9889300b449: Time spent starting tablet: real 0.015s user 0.005s sys 0.011s
W20260809 12:17:26.474136 31616 tablet.cc:2369] T 747d79a25a8e45abb83ef0a493ed475d P 918856a3d1054b24b5c3410deca11388: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:17:26.504760 31831 tablet.cc:2369] T 747d79a25a8e45abb83ef0a493ed475d P 5f34776fb824418b99b2d9889300b449: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:17:26.678882 31410 tablet.cc:2369] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:17:26.788858 31873 raft_consensus.cc:482] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:17:26.789238 31873 raft_consensus.cc:504] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } }
I20260809 12:17:26.791229 31873 leader_election.cc:288] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5f34776fb824418b99b2d9889300b449 (127.19.183.195:35519), 918856a3d1054b24b5c3410deca11388 (127.19.183.194:43999)
I20260809 12:17:26.798494 31856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:26.791685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:49221 (local address 127.19.183.195:35519)
0809 12:17:26.791871 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:26.791878 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:26.792480 (+ 602us) server_negotiation.cc:407] Connection header received
0809 12:17:26.792712 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:26.792720 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:26.792816 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:26.792919 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:17:26.793615 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.794536 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.796628 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.797510 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.797637 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:26.797689 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:26.798103 (+ 414us) server_negotiation.cc:299] Negotiation successful
0809 12:17:26.798242 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 12:17:26.798576 31877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:26.791567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:35519 (local address 127.19.183.193:49221)
0809 12:17:26.792355 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:26.792375 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:26.792559 (+ 184us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:26.792985 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:26.792998 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:26.793037 (+ 39us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:17:26.793432 (+ 395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:26.793446 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.794683 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.794689 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:26.796472 (+ 1783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:26.796484 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.797735 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.797742 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:26.797923 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:26.797961 (+ 38us) client_negotiation.cc:769] Sending connection context
0809 12:17:26.798071 (+ 110us) client_negotiation.cc:240] Negotiation successful
0809 12:17:26.798161 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"mutex_wait_us":54,"thread_start_us":466,"threads_started":1}
I20260809 12:17:26.798748 31858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:26.791669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:42077 (local address 127.19.183.194:43999)
0809 12:17:26.791871 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:26.791878 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:26.793252 (+ 1374us) server_negotiation.cc:407] Connection header received
0809 12:17:26.793396 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:26.793401 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:26.793486 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:26.793580 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:17:26.794188 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.795152 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.796948 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.797770 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.797938 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:26.798011 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:26.798341 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 12:17:26.798466 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 12:17:26.799211 31878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:26.791778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:43999 (local address 127.19.183.193:42077)
0809 12:17:26.793131 (+ 1353us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:26.793148 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:26.793255 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:26.793615 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:26.793628 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:26.793653 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:17:26.794031 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:26.794042 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.795292 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.795296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:26.796786 (+ 1490us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:26.796801 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.797977 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.797982 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:26.798157 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:26.798190 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 12:17:26.798291 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 12:17:26.798413 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1220,"mutex_wait_us":1,"thread_start_us":1026,"threads_started":1}
I20260809 12:17:26.799693 31750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "747d79a25a8e45abb83ef0a493ed475d" candidate_uuid: "6f753bec541d42d9bcf7da08421aae48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f34776fb824418b99b2d9889300b449" is_pre_election: true
I20260809 12:17:26.800174 31750 raft_consensus.cc:2454] T 747d79a25a8e45abb83ef0a493ed475d P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f753bec541d42d9bcf7da08421aae48 in term 0.
I20260809 12:17:26.800302 31553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "747d79a25a8e45abb83ef0a493ed475d" candidate_uuid: "6f753bec541d42d9bcf7da08421aae48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918856a3d1054b24b5c3410deca11388" is_pre_election: true
I20260809 12:17:26.800855 31219 leader_election.cc:302] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f34776fb824418b99b2d9889300b449, 6f753bec541d42d9bcf7da08421aae48; no voters:
I20260809 12:17:26.800917 31553 raft_consensus.cc:2454] T 747d79a25a8e45abb83ef0a493ed475d P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f753bec541d42d9bcf7da08421aae48 in term 0.
I20260809 12:17:26.801271 31873 raft_consensus.cc:2781] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:17:26.801450 31873 raft_consensus.cc:482] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:17:26.801558 31873 raft_consensus.cc:3037] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:26.803877 31873 raft_consensus.cc:504] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } }
I20260809 12:17:26.804600 31873 leader_election.cc:288] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48 [CANDIDATE]: Term 1 election: Requested vote from peers 5f34776fb824418b99b2d9889300b449 (127.19.183.195:35519), 918856a3d1054b24b5c3410deca11388 (127.19.183.194:43999)
I20260809 12:17:26.804924 31750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "747d79a25a8e45abb83ef0a493ed475d" candidate_uuid: "6f753bec541d42d9bcf7da08421aae48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f34776fb824418b99b2d9889300b449"
I20260809 12:17:26.805130 31750 raft_consensus.cc:3037] T 747d79a25a8e45abb83ef0a493ed475d P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:26.805183 31553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "747d79a25a8e45abb83ef0a493ed475d" candidate_uuid: "6f753bec541d42d9bcf7da08421aae48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918856a3d1054b24b5c3410deca11388"
I20260809 12:17:26.805387 31553 raft_consensus.cc:3037] T 747d79a25a8e45abb83ef0a493ed475d P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:26.808245 31750 raft_consensus.cc:2454] T 747d79a25a8e45abb83ef0a493ed475d P 5f34776fb824418b99b2d9889300b449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f753bec541d42d9bcf7da08421aae48 in term 1.
I20260809 12:17:26.808560 31553 raft_consensus.cc:2454] T 747d79a25a8e45abb83ef0a493ed475d P 918856a3d1054b24b5c3410deca11388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f753bec541d42d9bcf7da08421aae48 in term 1.
I20260809 12:17:26.808636 31219 leader_election.cc:302] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f34776fb824418b99b2d9889300b449, 6f753bec541d42d9bcf7da08421aae48; no voters:
I20260809 12:17:26.808916 31873 raft_consensus.cc:2781] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:17:26.809381 31873 raft_consensus.cc:686] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48 [term 1 LEADER]: Becoming Leader. State: Replica: 6f753bec541d42d9bcf7da08421aae48, State: Running, Role: LEADER
I20260809 12:17:26.809824 31873 consensus_queue.cc:237] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } }
I20260809 12:17:26.814545 31090 catalog_manager.cc:5495] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f753bec541d42d9bcf7da08421aae48 (127.19.183.193). New cstate: current_term: 1 leader_uuid: "6f753bec541d42d9bcf7da08421aae48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } health_report { overall_health: HEALTHY } } }
I20260809 12:17:26.839649 31859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:26.834833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43416 (local address 127.19.183.193:42711)
0809 12:17:26.835000 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:26.835004 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:26.835020 (+ 16us) server_negotiation.cc:407] Connection header received
0809 12:17:26.835215 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:26.835218 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:26.835298 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:26.835434 (+ 136us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:17:26.835739 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.836399 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.837641 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:26.838019 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:26.838107 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:26.838472 (+ 365us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:17:26.839025 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:17:26.839132 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:26.839228 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 12:17:26.839320 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 12:17:26.848469 31750 raft_consensus.cc:1261] T 747d79a25a8e45abb83ef0a493ed475d P 5f34776fb824418b99b2d9889300b449 [term 1 FOLLOWER]: Refusing update from remote peer 6f753bec541d42d9bcf7da08421aae48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:17:26.848471 31553 raft_consensus.cc:1261] T 747d79a25a8e45abb83ef0a493ed475d P 918856a3d1054b24b5c3410deca11388 [term 1 FOLLOWER]: Refusing update from remote peer 6f753bec541d42d9bcf7da08421aae48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:17:26.849473 31879 consensus_queue.cc:1035] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:17:26.849833 31873 consensus_queue.cc:1035] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:17:26.861049 31885 mvcc.cc:205] Tried to move back new op lower bound from 7316594060681474048 to 7316594060536688640. Current Snapshot: MvccSnapshot[applied={T|T < 7316594060681474048}]
I20260809 12:17:26.876072 31091 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:17:26.876314 31091 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:17:26.879861 31091 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112: Sending DeleteTablet for 3 replicas of tablet 747d79a25a8e45abb83ef0a493ed475d
I20260809 12:17:26.880990 31514 tablet_service.cc:1514] Processing DeleteTablet for tablet 747d79a25a8e45abb83ef0a493ed475d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:17:26 UTC) from {username='oryx', principal='oryx/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:34530
I20260809 12:17:26.881042 31717 tablet_service.cc:1514] Processing DeleteTablet for tablet 747d79a25a8e45abb83ef0a493ed475d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:17:26 UTC) from {username='oryx', principal='oryx/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:50392
I20260809 12:17:26.881217 31290 tablet_service.cc:1514] Processing DeleteTablet for tablet 747d79a25a8e45abb83ef0a493ed475d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:17:26 UTC) from {username='oryx', principal='oryx/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:43402
I20260809 12:17:26.882094 31897 tablet_replica.cc:330] T 747d79a25a8e45abb83ef0a493ed475d P 5f34776fb824418b99b2d9889300b449: stopping tablet replica
I20260809 12:17:26.882652 31897 raft_consensus.cc:2229] T 747d79a25a8e45abb83ef0a493ed475d P 5f34776fb824418b99b2d9889300b449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:17:26.883266 31897 raft_consensus.cc:2258] T 747d79a25a8e45abb83ef0a493ed475d P 5f34776fb824418b99b2d9889300b449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:17:26.883730 31091 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53160:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:17:26.884568 31091 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:17:26.885896 31897 ts_tablet_manager.cc:1905] T 747d79a25a8e45abb83ef0a493ed475d P 5f34776fb824418b99b2d9889300b449: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:17:26.893720 31897 ts_tablet_manager.cc:1918] T 747d79a25a8e45abb83ef0a493ed475d P 5f34776fb824418b99b2d9889300b449: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:17:26.893895 31897 log.cc:1196] T 747d79a25a8e45abb83ef0a493ed475d P 5f34776fb824418b99b2d9889300b449: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/747d79a25a8e45abb83ef0a493ed475d
I20260809 12:17:26.894408 31897 ts_tablet_manager.cc:1939] T 747d79a25a8e45abb83ef0a493ed475d P 5f34776fb824418b99b2d9889300b449: Deleting consensus metadata
I20260809 12:17:26.895295 31900 tablet_replica.cc:330] T 747d79a25a8e45abb83ef0a493ed475d P 918856a3d1054b24b5c3410deca11388: stopping tablet replica
I20260809 12:17:26.895716 31717 tablet_service.cc:1467] Processing CreateTablet for tablet 37ae5c73e3814dd89dda6c658c8f77c4 (DEFAULT_TABLE table=test-table [id=29dd438d44154177b2132067c70f45c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:17:26.895810 31900 raft_consensus.cc:2229] T 747d79a25a8e45abb83ef0a493ed475d P 918856a3d1054b24b5c3410deca11388 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:17:26.895885 31288 tablet_service.cc:1467] Processing CreateTablet for tablet 37ae5c73e3814dd89dda6c658c8f77c4 (DEFAULT_TABLE table=test-table [id=29dd438d44154177b2132067c70f45c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:17:26.896168 31717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37ae5c73e3814dd89dda6c658c8f77c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:26.896368 31900 raft_consensus.cc:2258] T 747d79a25a8e45abb83ef0a493ed475d P 918856a3d1054b24b5c3410deca11388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:17:26.896453 31288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37ae5c73e3814dd89dda6c658c8f77c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:26.898962 31900 ts_tablet_manager.cc:1905] T 747d79a25a8e45abb83ef0a493ed475d P 918856a3d1054b24b5c3410deca11388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:17:26.902230 31867 tablet_bootstrap.cc:492] T 37ae5c73e3814dd89dda6c658c8f77c4 P 5f34776fb824418b99b2d9889300b449: Bootstrap starting.
I20260809 12:17:26.903749 31901 tablet_replica.cc:330] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48: stopping tablet replica
I20260809 12:17:26.903813 31867 tablet_bootstrap.cc:654] T 37ae5c73e3814dd89dda6c658c8f77c4 P 5f34776fb824418b99b2d9889300b449: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:26.904139 31901 raft_consensus.cc:2229] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:17:26.904789 31901 raft_consensus.cc:2258] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:17:26.905946 31900 ts_tablet_manager.cc:1918] T 747d79a25a8e45abb83ef0a493ed475d P 918856a3d1054b24b5c3410deca11388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:17:26.906106 31900 log.cc:1196] T 747d79a25a8e45abb83ef0a493ed475d P 918856a3d1054b24b5c3410deca11388: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/747d79a25a8e45abb83ef0a493ed475d
I20260809 12:17:26.906177 31864 tablet_bootstrap.cc:492] T 37ae5c73e3814dd89dda6c658c8f77c4 P 6f753bec541d42d9bcf7da08421aae48: Bootstrap starting.
I20260809 12:17:26.906638 31900 ts_tablet_manager.cc:1939] T 747d79a25a8e45abb83ef0a493ed475d P 918856a3d1054b24b5c3410deca11388: Deleting consensus metadata
I20260809 12:17:26.907023 31901 ts_tablet_manager.cc:1905] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:17:26.908386 31864 tablet_bootstrap.cc:654] T 37ae5c73e3814dd89dda6c658c8f77c4 P 6f753bec541d42d9bcf7da08421aae48: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:26.910234 31076 catalog_manager.cc:4841] TS 5f34776fb824418b99b2d9889300b449 (127.19.183.195:35519): tablet 747d79a25a8e45abb83ef0a493ed475d (table test-table [id=360ce4ef8e82417ca02785f2ef7a3a89]) successfully deleted
I20260809 12:17:26.910810 31073 catalog_manager.cc:4841] TS 918856a3d1054b24b5c3410deca11388 (127.19.183.194:43999): tablet 747d79a25a8e45abb83ef0a493ed475d (table test-table [id=360ce4ef8e82417ca02785f2ef7a3a89]) successfully deleted
I20260809 12:17:26.913415 31867 tablet_bootstrap.cc:492] T 37ae5c73e3814dd89dda6c658c8f77c4 P 5f34776fb824418b99b2d9889300b449: No bootstrap required, opened a new log
I20260809 12:17:26.913591 31867 ts_tablet_manager.cc:1397] T 37ae5c73e3814dd89dda6c658c8f77c4 P 5f34776fb824418b99b2d9889300b449: Time spent bootstrapping tablet: real 0.011s user 0.008s sys 0.001s
I20260809 12:17:26.914279 31901 ts_tablet_manager.cc:1918] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:17:26.914448 31901 log.cc:1196] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/747d79a25a8e45abb83ef0a493ed475d
I20260809 12:17:26.914470 31867 raft_consensus.cc:348] T 37ae5c73e3814dd89dda6c658c8f77c4 P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:26.914758 31867 raft_consensus.cc:374] T 37ae5c73e3814dd89dda6c658c8f77c4 P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:26.914920 31867 raft_consensus.cc:729] T 37ae5c73e3814dd89dda6c658c8f77c4 P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f34776fb824418b99b2d9889300b449, State: Initialized, Role: FOLLOWER
I20260809 12:17:26.915035 31901 ts_tablet_manager.cc:1939] T 747d79a25a8e45abb83ef0a493ed475d P 6f753bec541d42d9bcf7da08421aae48: Deleting consensus metadata
I20260809 12:17:26.915263 31867 consensus_queue.cc:260] T 37ae5c73e3814dd89dda6c658c8f77c4 P 5f34776fb824418b99b2d9889300b449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:26.916836 31867 ts_tablet_manager.cc:1428] T 37ae5c73e3814dd89dda6c658c8f77c4 P 5f34776fb824418b99b2d9889300b449: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 12:17:26.916841 31072 catalog_manager.cc:4841] TS 6f753bec541d42d9bcf7da08421aae48 (127.19.183.193:42711): tablet 747d79a25a8e45abb83ef0a493ed475d (table test-table [id=360ce4ef8e82417ca02785f2ef7a3a89]) successfully deleted
I20260809 12:17:26.918022 31514 tablet_service.cc:1467] Processing CreateTablet for tablet 37ae5c73e3814dd89dda6c658c8f77c4 (DEFAULT_TABLE table=test-table [id=29dd438d44154177b2132067c70f45c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:17:26.918485 31514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37ae5c73e3814dd89dda6c658c8f77c4. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:26.924371 31864 tablet_bootstrap.cc:492] T 37ae5c73e3814dd89dda6c658c8f77c4 P 6f753bec541d42d9bcf7da08421aae48: No bootstrap required, opened a new log
I20260809 12:17:26.924690 31864 ts_tablet_manager.cc:1397] T 37ae5c73e3814dd89dda6c658c8f77c4 P 6f753bec541d42d9bcf7da08421aae48: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.004s
I20260809 12:17:26.925062 31865 tablet_bootstrap.cc:492] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388: Bootstrap starting.
I20260809 12:17:26.925736 31864 raft_consensus.cc:348] T 37ae5c73e3814dd89dda6c658c8f77c4 P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:26.925997 31864 raft_consensus.cc:374] T 37ae5c73e3814dd89dda6c658c8f77c4 P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:26.926090 31864 raft_consensus.cc:729] T 37ae5c73e3814dd89dda6c658c8f77c4 P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f753bec541d42d9bcf7da08421aae48, State: Initialized, Role: FOLLOWER
I20260809 12:17:26.926362 31864 consensus_queue.cc:260] T 37ae5c73e3814dd89dda6c658c8f77c4 P 6f753bec541d42d9bcf7da08421aae48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:26.926762 31865 tablet_bootstrap.cc:654] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:26.927784 31864 ts_tablet_manager.cc:1428] T 37ae5c73e3814dd89dda6c658c8f77c4 P 6f753bec541d42d9bcf7da08421aae48: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 12:17:26.929232 31865 tablet_bootstrap.cc:492] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388: No bootstrap required, opened a new log
I20260809 12:17:26.929340 31865 ts_tablet_manager.cc:1397] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260809 12:17:26.930140 31865 raft_consensus.cc:348] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:26.930302 31865 raft_consensus.cc:374] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:26.930419 31865 raft_consensus.cc:729] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 918856a3d1054b24b5c3410deca11388, State: Initialized, Role: FOLLOWER
I20260809 12:17:26.930692 31865 consensus_queue.cc:260] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:26.931272 31865 ts_tablet_manager.cc:1428] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 12:17:27.046423 31872 raft_consensus.cc:482] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:17:27.046679 31872 raft_consensus.cc:504] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.048636 31872 leader_election.cc:288] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f753bec541d42d9bcf7da08421aae48 (127.19.183.193:42711), 5f34776fb824418b99b2d9889300b449 (127.19.183.195:35519)
I20260809 12:17:27.056974 31859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:27.049324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:56585 (local address 127.19.183.193:42711)
0809 12:17:27.049617 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:27.049626 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:27.050367 (+ 741us) server_negotiation.cc:407] Connection header received
0809 12:17:27.050681 (+ 314us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:27.050692 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:27.050796 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:27.050880 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:17:27.051609 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.052827 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.055181 (+ 2354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.055948 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.056137 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:27.056220 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:27.056494 (+ 274us) server_negotiation.cc:299] Negotiation successful
0809 12:17:27.056640 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 12:17:27.056984 31907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:27.049072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:42711 (local address 127.19.183.194:56585)
0809 12:17:27.050257 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:27.050277 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:27.050485 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:27.050946 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:27.050950 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:27.050978 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:17:27.051411 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:27.051430 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.052985 (+ 1555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.052990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:27.054999 (+ 2009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:27.055007 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.056175 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.056181 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:27.056335 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:27.056359 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 12:17:27.056465 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 12:17:27.056590 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":979,"thread_start_us":732,"threads_started":1}
I20260809 12:17:27.057920 31856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:27.049353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:51037 (local address 127.19.183.195:35519)
0809 12:17:27.049545 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:27.049551 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:27.050743 (+ 1192us) server_negotiation.cc:407] Connection header received
0809 12:17:27.050906 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:27.050911 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:27.051010 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:27.051104 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:17:27.051795 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.052557 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.055167 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.056385 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.056532 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:27.056601 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:27.057503 (+ 902us) server_negotiation.cc:299] Negotiation successful
0809 12:17:27.057654 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:17:27.057847 31325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37ae5c73e3814dd89dda6c658c8f77c4" candidate_uuid: "918856a3d1054b24b5c3410deca11388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f753bec541d42d9bcf7da08421aae48" is_pre_election: true
I20260809 12:17:27.057963 31908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:27.049093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:35519 (local address 127.19.183.194:51037)
0809 12:17:27.050606 (+ 1513us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:27.050630 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:27.050780 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:27.051135 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:27.051140 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:27.051161 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:17:27.051624 (+ 463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:27.051634 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.052703 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.052708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:27.054970 (+ 2262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:27.054987 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.056768 (+ 1781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.056775 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:27.056982 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:27.057015 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 12:17:27.057453 (+ 438us) client_negotiation.cc:240] Negotiation successful
0809 12:17:27.057585 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1383,"mutex_wait_us":553,"thread_start_us":580,"threads_started":1}
I20260809 12:17:27.058358 31325 raft_consensus.cc:2454] T 37ae5c73e3814dd89dda6c658c8f77c4 P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 918856a3d1054b24b5c3410deca11388 in term 0.
I20260809 12:17:27.059052 31437 leader_election.cc:302] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f753bec541d42d9bcf7da08421aae48, 918856a3d1054b24b5c3410deca11388; no voters:
I20260809 12:17:27.059104 31750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37ae5c73e3814dd89dda6c658c8f77c4" candidate_uuid: "918856a3d1054b24b5c3410deca11388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f34776fb824418b99b2d9889300b449" is_pre_election: true
I20260809 12:17:27.059353 31750 raft_consensus.cc:2454] T 37ae5c73e3814dd89dda6c658c8f77c4 P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 918856a3d1054b24b5c3410deca11388 in term 0.
I20260809 12:17:27.059448 31872 raft_consensus.cc:2781] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:17:27.059545 31872 raft_consensus.cc:482] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:17:27.059656 31872 raft_consensus.cc:3037] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:27.061959 31872 raft_consensus.cc:504] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.062587 31872 leader_election.cc:288] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388 [CANDIDATE]: Term 1 election: Requested vote from peers 6f753bec541d42d9bcf7da08421aae48 (127.19.183.193:42711), 5f34776fb824418b99b2d9889300b449 (127.19.183.195:35519)
I20260809 12:17:27.063053 31325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37ae5c73e3814dd89dda6c658c8f77c4" candidate_uuid: "918856a3d1054b24b5c3410deca11388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f753bec541d42d9bcf7da08421aae48"
I20260809 12:17:27.063159 31750 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37ae5c73e3814dd89dda6c658c8f77c4" candidate_uuid: "918856a3d1054b24b5c3410deca11388" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f34776fb824418b99b2d9889300b449"
I20260809 12:17:27.063273 31325 raft_consensus.cc:3037] T 37ae5c73e3814dd89dda6c658c8f77c4 P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:27.063372 31750 raft_consensus.cc:3037] T 37ae5c73e3814dd89dda6c658c8f77c4 P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:27.066066 31325 raft_consensus.cc:2454] T 37ae5c73e3814dd89dda6c658c8f77c4 P 6f753bec541d42d9bcf7da08421aae48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 918856a3d1054b24b5c3410deca11388 in term 1.
I20260809 12:17:27.066303 31750 raft_consensus.cc:2454] T 37ae5c73e3814dd89dda6c658c8f77c4 P 5f34776fb824418b99b2d9889300b449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 918856a3d1054b24b5c3410deca11388 in term 1.
I20260809 12:17:27.066505 31437 leader_election.cc:302] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f753bec541d42d9bcf7da08421aae48, 918856a3d1054b24b5c3410deca11388; no voters:
I20260809 12:17:27.066833 31872 raft_consensus.cc:2781] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:17:27.067400 31872 raft_consensus.cc:686] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388 [term 1 LEADER]: Becoming Leader. State: Replica: 918856a3d1054b24b5c3410deca11388, State: Running, Role: LEADER
I20260809 12:17:27.067806 31872 consensus_queue.cc:237] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.071458 31091 catalog_manager.cc:5495] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388 reported cstate change: term changed from 0 to 1, leader changed from <none> to 918856a3d1054b24b5c3410deca11388 (127.19.183.194). New cstate: current_term: 1 leader_uuid: "918856a3d1054b24b5c3410deca11388" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } health_report { overall_health: UNKNOWN } } }
Aug 09 12:17:27 dist-test-slave-39l8 krb5kdc[31025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277844, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.254@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM
I20260809 12:17:27.143375 31918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:27.129186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:43903 (local address 127.0.0.1:53172)
0809 12:17:27.129920 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:27.129943 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:27.130128 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:27.130473 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:27.130479 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:27.131203 (+ 724us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:27.131422 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:27.131433 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.132553 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.132560 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:27.133506 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:27.133518 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.134236 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.134242 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:27.134404 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:27.135148 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:27.135171 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:27.137259 (+ 2088us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:27.140526 (+ 3267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:27.140539 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:27.140546 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:27.141112 (+ 566us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:27.141479 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:27.141484 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:27.141487 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:27.141656 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:27.142187 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:27.142195 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:27.142369 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 12:17:27.142537 (+ 168us) client_negotiation.cc:240] Negotiation successful
0809 12:17:27.142888 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":405,"threads_started":1}
I20260809 12:17:27.143404 31919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:27.129222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53172 (local address 127.19.183.254:43903)
0809 12:17:27.130199 (+ 977us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:27.130206 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:27.130229 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:17:27.130282 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:27.130285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:27.130332 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:27.130435 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:27.131587 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.132403 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.133651 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.134054 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.134214 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:27.137464 (+ 3250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:27.137469 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:27.137472 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:27.137499 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:27.140408 (+ 2909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:27.141226 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:27.141234 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:27.141237 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:27.141360 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:27.141798 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:27.141802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:27.141804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:27.142049 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:27.142213 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:27.142683 (+ 470us) server_negotiation.cc:299] Negotiation successful
0809 12:17:27.143008 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":111,"server-negotiator.queue_time_us":862,"thread_start_us":678,"threads_started":1}
I20260809 12:17:27.148905 31091 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:53172:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 12:17:27.149977 31091 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:17:27.156970 31288 tablet_service.cc:1467] Processing CreateTablet for tablet a2ad790ae4a342eda00b919a03d9392b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=92ea6016be2c41e7b645e5f0585afaad]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:17:27.157055 31514 tablet_service.cc:1467] Processing CreateTablet for tablet a2ad790ae4a342eda00b919a03d9392b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=92ea6016be2c41e7b645e5f0585afaad]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:17:27.157207 31717 tablet_service.cc:1467] Processing CreateTablet for tablet a2ad790ae4a342eda00b919a03d9392b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=92ea6016be2c41e7b645e5f0585afaad]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 12:17:27.157555 31288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2ad790ae4a342eda00b919a03d9392b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:27.157665 31514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2ad790ae4a342eda00b919a03d9392b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:27.157742 31717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a2ad790ae4a342eda00b919a03d9392b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:27.165037 31867 tablet_bootstrap.cc:492] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449: Bootstrap starting.
I20260809 12:17:27.166610 31867 tablet_bootstrap.cc:654] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:27.167660 31865 tablet_bootstrap.cc:492] T a2ad790ae4a342eda00b919a03d9392b P 918856a3d1054b24b5c3410deca11388: Bootstrap starting.
I20260809 12:17:27.168965 31865 tablet_bootstrap.cc:654] T a2ad790ae4a342eda00b919a03d9392b P 918856a3d1054b24b5c3410deca11388: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:27.170948 31867 tablet_bootstrap.cc:492] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449: No bootstrap required, opened a new log
I20260809 12:17:27.171131 31867 ts_tablet_manager.cc:1397] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449: Time spent bootstrapping tablet: real 0.006s user 0.003s sys 0.000s
I20260809 12:17:27.171723 31867 raft_consensus.cc:348] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.171949 31867 raft_consensus.cc:374] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:27.172055 31867 raft_consensus.cc:729] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f34776fb824418b99b2d9889300b449, State: Initialized, Role: FOLLOWER
I20260809 12:17:27.172909 31865 tablet_bootstrap.cc:492] T a2ad790ae4a342eda00b919a03d9392b P 918856a3d1054b24b5c3410deca11388: No bootstrap required, opened a new log
I20260809 12:17:27.173082 31865 ts_tablet_manager.cc:1397] T a2ad790ae4a342eda00b919a03d9392b P 918856a3d1054b24b5c3410deca11388: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.001s
I20260809 12:17:27.173655 31865 raft_consensus.cc:348] T a2ad790ae4a342eda00b919a03d9392b P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.173925 31865 raft_consensus.cc:374] T a2ad790ae4a342eda00b919a03d9392b P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:27.174007 31865 raft_consensus.cc:729] T a2ad790ae4a342eda00b919a03d9392b P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 918856a3d1054b24b5c3410deca11388, State: Initialized, Role: FOLLOWER
I20260809 12:17:27.174620 31865 consensus_queue.cc:260] T a2ad790ae4a342eda00b919a03d9392b P 918856a3d1054b24b5c3410deca11388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.175021 31867 consensus_queue.cc:260] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.175437 31872 tablet_replica.cc:439] T a2ad790ae4a342eda00b919a03d9392b P 918856a3d1054b24b5c3410deca11388: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } } }
I20260809 12:17:27.175726 31872 tablet_replica.cc:442] T a2ad790ae4a342eda00b919a03d9392b P 918856a3d1054b24b5c3410deca11388: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:17:27.176141 31865 ts_tablet_manager.cc:1428] T a2ad790ae4a342eda00b919a03d9392b P 918856a3d1054b24b5c3410deca11388: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 12:17:27.176573 31875 tablet_replica.cc:439] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } } }
I20260809 12:17:27.176954 31875 tablet_replica.cc:442] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:17:27.177457 31867 ts_tablet_manager.cc:1428] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449: Time spent starting tablet: real 0.006s user 0.001s sys 0.000s
I20260809 12:17:27.177810 31864 tablet_bootstrap.cc:492] T a2ad790ae4a342eda00b919a03d9392b P 6f753bec541d42d9bcf7da08421aae48: Bootstrap starting.
I20260809 12:17:27.180693 31864 tablet_bootstrap.cc:654] T a2ad790ae4a342eda00b919a03d9392b P 6f753bec541d42d9bcf7da08421aae48: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:27.187664 31864 tablet_bootstrap.cc:492] T a2ad790ae4a342eda00b919a03d9392b P 6f753bec541d42d9bcf7da08421aae48: No bootstrap required, opened a new log
I20260809 12:17:27.187875 31864 ts_tablet_manager.cc:1397] T a2ad790ae4a342eda00b919a03d9392b P 6f753bec541d42d9bcf7da08421aae48: Time spent bootstrapping tablet: real 0.010s user 0.000s sys 0.003s
I20260809 12:17:27.188558 31864 raft_consensus.cc:348] T a2ad790ae4a342eda00b919a03d9392b P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.188813 31864 raft_consensus.cc:374] T a2ad790ae4a342eda00b919a03d9392b P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:27.188911 31864 raft_consensus.cc:729] T a2ad790ae4a342eda00b919a03d9392b P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f753bec541d42d9bcf7da08421aae48, State: Initialized, Role: FOLLOWER
I20260809 12:17:27.189179 31864 consensus_queue.cc:260] T a2ad790ae4a342eda00b919a03d9392b P 6f753bec541d42d9bcf7da08421aae48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.189859 31864 ts_tablet_manager.cc:1428] T a2ad790ae4a342eda00b919a03d9392b P 6f753bec541d42d9bcf7da08421aae48: Time spent starting tablet: real 0.002s user 0.000s sys 0.000s
I20260809 12:17:27.189626 31891 tablet_replica.cc:439] T a2ad790ae4a342eda00b919a03d9392b P 6f753bec541d42d9bcf7da08421aae48: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } } }
I20260809 12:17:27.190150 31891 tablet_replica.cc:442] T a2ad790ae4a342eda00b919a03d9392b P 6f753bec541d42d9bcf7da08421aae48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:17:27.203487 31875 raft_consensus.cc:482] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:17:27.203718 31875 raft_consensus.cc:504] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.205471 31875 leader_election.cc:288] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6f753bec541d42d9bcf7da08421aae48 (127.19.183.193:42711), 918856a3d1054b24b5c3410deca11388 (127.19.183.194:43999)
I20260809 12:17:27.217343 31859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:27.206194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:53363 (local address 127.19.183.193:42711)
0809 12:17:27.206698 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:27.206725 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:27.210308 (+ 3583us) server_negotiation.cc:407] Connection header received
0809 12:17:27.210729 (+ 421us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:27.210735 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:27.210815 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:27.210910 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:17:27.211604 (+ 694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.212810 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.215339 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.216211 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.216390 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:27.216502 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:27.216855 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 12:17:27.217000 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298}
I20260809 12:17:27.217602 31858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:27.206414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:55141 (local address 127.19.183.194:43999)
0809 12:17:27.206903 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:27.206911 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:27.210756 (+ 3845us) server_negotiation.cc:407] Connection header received
0809 12:17:27.210928 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:27.210934 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:27.211012 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:27.211115 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:17:27.211793 (+ 678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.212561 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.215260 (+ 2699us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.216527 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.216707 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:27.216798 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:27.217143 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 12:17:27.217295 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333}
I20260809 12:17:27.218405 31926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:27.205852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:43999 (local address 127.19.183.195:55141)
0809 12:17:27.210581 (+ 4729us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:27.210623 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:27.210777 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:27.211132 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:27.211141 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:27.211161 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:17:27.211641 (+ 480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:27.211651 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.212707 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.212713 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:27.215045 (+ 2332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:27.215062 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.216783 (+ 1721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.216790 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:27.216954 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:27.216986 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 12:17:27.217840 (+ 854us) client_negotiation.cc:240] Negotiation successful
0809 12:17:27.217980 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4622,"mutex_wait_us":4,"thread_start_us":3986,"threads_started":1}
I20260809 12:17:27.218546 31927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:27.205715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:42711 (local address 127.19.183.195:53363)
0809 12:17:27.210186 (+ 4471us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:27.210212 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:27.210513 (+ 301us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:27.210936 (+ 423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:27.210940 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:27.210964 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:17:27.211376 (+ 412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:27.211387 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.212985 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.212991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:27.215179 (+ 2188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:27.215190 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.216405 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.216411 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:27.216597 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:27.216657 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 12:17:27.218038 (+ 1381us) client_negotiation.cc:240] Negotiation successful
0809 12:17:27.218184 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4278,"thread_start_us":4537,"threads_started":1}
I20260809 12:17:27.219218 31553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2ad790ae4a342eda00b919a03d9392b" candidate_uuid: "5f34776fb824418b99b2d9889300b449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918856a3d1054b24b5c3410deca11388" is_pre_election: true
I20260809 12:17:27.219254 31325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2ad790ae4a342eda00b919a03d9392b" candidate_uuid: "5f34776fb824418b99b2d9889300b449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f753bec541d42d9bcf7da08421aae48" is_pre_election: true
I20260809 12:17:27.219489 31553 raft_consensus.cc:2454] T a2ad790ae4a342eda00b919a03d9392b P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f34776fb824418b99b2d9889300b449 in term 0.
I20260809 12:17:27.219506 31325 raft_consensus.cc:2454] T a2ad790ae4a342eda00b919a03d9392b P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5f34776fb824418b99b2d9889300b449 in term 0.
I20260809 12:17:27.220016 31644 leader_election.cc:302] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5f34776fb824418b99b2d9889300b449, 6f753bec541d42d9bcf7da08421aae48, 918856a3d1054b24b5c3410deca11388; no voters:
I20260809 12:17:27.220441 31875 raft_consensus.cc:2781] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:17:27.220566 31875 raft_consensus.cc:482] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:17:27.220665 31875 raft_consensus.cc:3037] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:27.222993 31875 raft_consensus.cc:504] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.223881 31875 leader_election.cc:288] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449 [CANDIDATE]: Term 1 election: Requested vote from peers 6f753bec541d42d9bcf7da08421aae48 (127.19.183.193:42711), 918856a3d1054b24b5c3410deca11388 (127.19.183.194:43999)
I20260809 12:17:27.224170 31325 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2ad790ae4a342eda00b919a03d9392b" candidate_uuid: "5f34776fb824418b99b2d9889300b449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6f753bec541d42d9bcf7da08421aae48"
I20260809 12:17:27.224292 31553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a2ad790ae4a342eda00b919a03d9392b" candidate_uuid: "5f34776fb824418b99b2d9889300b449" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918856a3d1054b24b5c3410deca11388"
I20260809 12:17:27.224418 31325 raft_consensus.cc:3037] T a2ad790ae4a342eda00b919a03d9392b P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:27.224560 31553 raft_consensus.cc:3037] T a2ad790ae4a342eda00b919a03d9392b P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:27.226922 31325 raft_consensus.cc:2454] T a2ad790ae4a342eda00b919a03d9392b P 6f753bec541d42d9bcf7da08421aae48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f34776fb824418b99b2d9889300b449 in term 1.
I20260809 12:17:27.227340 31642 leader_election.cc:302] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f34776fb824418b99b2d9889300b449, 6f753bec541d42d9bcf7da08421aae48; no voters:
I20260809 12:17:27.227694 31875 raft_consensus.cc:2781] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:17:27.228272 31875 raft_consensus.cc:686] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449 [term 1 LEADER]: Becoming Leader. State: Replica: 5f34776fb824418b99b2d9889300b449, State: Running, Role: LEADER
I20260809 12:17:27.228761 31553 raft_consensus.cc:2454] T a2ad790ae4a342eda00b919a03d9392b P 918856a3d1054b24b5c3410deca11388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5f34776fb824418b99b2d9889300b449 in term 1.
I20260809 12:17:27.228739 31875 consensus_queue.cc:237] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.231765 31929 tablet_replica.cc:439] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449: TxnStatusTablet state changed. Reason: New leader 5f34776fb824418b99b2d9889300b449. Latest consensus state: current_term: 1 leader_uuid: "5f34776fb824418b99b2d9889300b449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } } }
I20260809 12:17:27.232182 31929 tablet_replica.cc:442] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:17:27.232880 31931 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 12:17:27.233099 31931 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 12:17:27.233253 31090 catalog_manager.cc:5495] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5f34776fb824418b99b2d9889300b449 (127.19.183.195). New cstate: current_term: 1 leader_uuid: "5f34776fb824418b99b2d9889300b449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } health_report { overall_health: HEALTHY } } }
I20260809 12:17:27.255344 31918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:27.248055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:35519 (local address 127.0.0.1:50400)
0809 12:17:27.248419 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:27.248436 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:27.248608 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:27.249134 (+ 526us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:27.249138 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:27.249150 (+ 12us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:17:27.249367 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:27.249376 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.250495 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.250501 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:27.252115 (+ 1614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:27.252132 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.253475 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.253483 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:27.253701 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:27.253770 (+ 69us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:17:27.254803 (+ 1033us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:17:27.254822 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 12:17:27.254893 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 12:17:27.255043 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200}
I20260809 12:17:27.256084 31856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:27.248032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50400 (local address 127.19.183.195:35519)
0809 12:17:27.248226 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:27.248232 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:27.248511 (+ 279us) server_negotiation.cc:407] Connection header received
0809 12:17:27.248809 (+ 298us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:27.248813 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:27.248914 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:27.249125 (+ 211us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:17:27.249529 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.250367 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.252383 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.253161 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.253333 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:27.253998 (+ 665us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:17:27.254605 (+ 607us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:17:27.254747 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:27.255650 (+ 903us) server_negotiation.cc:299] Negotiation successful
0809 12:17:27.255738 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:17:27.260586 31553 raft_consensus.cc:1261] T a2ad790ae4a342eda00b919a03d9392b P 918856a3d1054b24b5c3410deca11388 [term 1 FOLLOWER]: Refusing update from remote peer 5f34776fb824418b99b2d9889300b449: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:17:27.260885 31325 raft_consensus.cc:1261] T a2ad790ae4a342eda00b919a03d9392b P 6f753bec541d42d9bcf7da08421aae48 [term 1 FOLLOWER]: Refusing update from remote peer 5f34776fb824418b99b2d9889300b449: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:17:27.261518 31929 consensus_queue.cc:1035] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:17:27.262158 31929 consensus_queue.cc:1035] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:17:27.276726 31891 tablet_replica.cc:439] T a2ad790ae4a342eda00b919a03d9392b P 6f753bec541d42d9bcf7da08421aae48: TxnStatusTablet state changed. Reason: New leader 5f34776fb824418b99b2d9889300b449. Latest consensus state: current_term: 1 leader_uuid: "5f34776fb824418b99b2d9889300b449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } } }
I20260809 12:17:27.277033 31891 tablet_replica.cc:442] T a2ad790ae4a342eda00b919a03d9392b P 6f753bec541d42d9bcf7da08421aae48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:17:27.283818 31872 tablet_replica.cc:439] T a2ad790ae4a342eda00b919a03d9392b P 918856a3d1054b24b5c3410deca11388: TxnStatusTablet state changed. Reason: New leader 5f34776fb824418b99b2d9889300b449. Latest consensus state: current_term: 1 leader_uuid: "5f34776fb824418b99b2d9889300b449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } } }
I20260809 12:17:27.284225 31872 tablet_replica.cc:442] T a2ad790ae4a342eda00b919a03d9392b P 918856a3d1054b24b5c3410deca11388: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:17:27.295609 31875 tablet_replica.cc:439] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5f34776fb824418b99b2d9889300b449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } } }
I20260809 12:17:27.295980 31875 tablet_replica.cc:442] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:17:27.297653 31891 tablet_replica.cc:439] T a2ad790ae4a342eda00b919a03d9392b P 6f753bec541d42d9bcf7da08421aae48: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5f34776fb824418b99b2d9889300b449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } } }
I20260809 12:17:27.297972 31891 tablet_replica.cc:442] T a2ad790ae4a342eda00b919a03d9392b P 6f753bec541d42d9bcf7da08421aae48: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:17:27.300802 31858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:27.292582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34542 (local address 127.19.183.194:43999)
0809 12:17:27.292911 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:27.292917 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:27.292935 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:17:27.293030 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:27.293036 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:27.293175 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:27.293344 (+ 169us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:17:27.293981 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.295190 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.297194 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.297776 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.297912 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:27.298611 (+ 699us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:17:27.299367 (+ 756us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:17:27.299812 (+ 445us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:27.300408 (+ 596us) server_negotiation.cc:299] Negotiation successful
0809 12:17:27.300496 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200}
I20260809 12:17:27.304979 31929 tablet_replica.cc:439] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5f34776fb824418b99b2d9889300b449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } } }
I20260809 12:17:27.305243 31929 tablet_replica.cc:442] T a2ad790ae4a342eda00b919a03d9392b P 5f34776fb824418b99b2d9889300b449: This TxnStatusTablet replica's current role is: LEADER
I20260809 12:17:27.307623 31887 mvcc.cc:205] Tried to move back new op lower bound from 7316594062371188736 to 7316594062254612480. Current Snapshot: MvccSnapshot[applied={T|T < 7316594062371188736}]
I20260809 12:17:27.307955 31872 tablet_replica.cc:439] T a2ad790ae4a342eda00b919a03d9392b P 918856a3d1054b24b5c3410deca11388: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5f34776fb824418b99b2d9889300b449" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } } }
I20260809 12:17:27.308274 31872 tablet_replica.cc:442] T a2ad790ae4a342eda00b919a03d9392b P 918856a3d1054b24b5c3410deca11388: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 12:17:27.338438 31856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:27.333120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:41263 (local address 127.19.183.195:35519)
0809 12:17:27.333328 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:27.333336 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:27.333416 (+ 80us) server_negotiation.cc:407] Connection header received
0809 12:17:27.333729 (+ 313us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:27.333734 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:27.333798 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:27.333896 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:17:27.334395 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.335234 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.336642 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.337116 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.337262 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:27.337690 (+ 428us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:17:27.337798 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:17:27.337896 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:27.338052 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 12:17:27.338127 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 12:17:27.338508 31943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:27.332667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:35519 (local address 127.19.183.194:41263)
0809 12:17:27.333318 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:27.333338 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:27.333544 (+ 206us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:27.333915 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:27.333919 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:27.333930 (+ 11us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:17:27.334243 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:27.334252 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.335384 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.335392 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:27.336488 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:27.336499 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.337298 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.337305 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:27.337438 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:27.337495 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:17:27.337902 (+ 407us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:17:27.337927 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 12:17:27.338023 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 12:17:27.338153 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":454,"thread_start_us":682,"threads_started":1}
I20260809 12:17:27.350589 31750 raft_consensus.cc:1261] T 37ae5c73e3814dd89dda6c658c8f77c4 P 5f34776fb824418b99b2d9889300b449 [term 1 FOLLOWER]: Refusing update from remote peer 918856a3d1054b24b5c3410deca11388: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:17:27.350922 31325 raft_consensus.cc:1261] T 37ae5c73e3814dd89dda6c658c8f77c4 P 6f753bec541d42d9bcf7da08421aae48 [term 1 FOLLOWER]: Refusing update from remote peer 918856a3d1054b24b5c3410deca11388: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:17:27.351516 31910 consensus_queue.cc:1035] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:17:27.352201 31910 consensus_queue.cc:1035] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:17:27.415668 31858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:27.408671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:33481 (local address 127.19.183.194:43999)
0809 12:17:27.408978 (+ 307us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:27.408999 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:27.409524 (+ 525us) server_negotiation.cc:407] Connection header received
0809 12:17:27.409767 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:27.409773 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:27.409882 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:27.409997 (+ 115us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:17:27.410623 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.411680 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.413777 (+ 2097us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.414232 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.414327 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:27.414951 (+ 624us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:17:27.415059 (+ 108us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:17:27.415146 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:27.415305 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 12:17:27.415367 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146}
I20260809 12:17:27.415858 31956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:27.408657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:43999 (local address 127.19.183.195:33481)
0809 12:17:27.409412 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:27.409430 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:27.409579 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:27.410020 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:27.410024 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:27.410034 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:17:27.410467 (+ 433us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:27.410478 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.411810 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.411816 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:27.413602 (+ 1786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:27.413627 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.414428 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.414435 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:27.414665 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:27.414769 (+ 104us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:17:27.415174 (+ 405us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:17:27.415186 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 12:17:27.415274 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 12:17:27.415421 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":451,"threads_started":1}
I20260809 12:17:27.461262 31091 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:17:27.461534 31091 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:53160:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:17:27.464908 31091 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112: Sending DeleteTablet for 3 replicas of tablet 37ae5c73e3814dd89dda6c658c8f77c4
I20260809 12:17:27.465698 31288 tablet_service.cc:1514] Processing DeleteTablet for tablet 37ae5c73e3814dd89dda6c658c8f77c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:17:27 UTC) from {username='oryx', principal='oryx/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:43402
I20260809 12:17:27.466795 31514 tablet_service.cc:1514] Processing DeleteTablet for tablet 37ae5c73e3814dd89dda6c658c8f77c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:17:27 UTC) from {username='oryx', principal='oryx/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:34530
I20260809 12:17:27.470934 31717 tablet_service.cc:1514] Processing DeleteTablet for tablet 37ae5c73e3814dd89dda6c658c8f77c4 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:17:27 UTC) from {username='oryx', principal='oryx/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:50392
I20260809 12:17:27.499920 31967 tablet_replica.cc:330] T 37ae5c73e3814dd89dda6c658c8f77c4 P 5f34776fb824418b99b2d9889300b449: stopping tablet replica
I20260809 12:17:27.500496 31963 tablet_replica.cc:330] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388: stopping tablet replica
I20260809 12:17:27.500784 31967 raft_consensus.cc:2229] T 37ae5c73e3814dd89dda6c658c8f77c4 P 5f34776fb824418b99b2d9889300b449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:17:27.501170 31967 raft_consensus.cc:2258] T 37ae5c73e3814dd89dda6c658c8f77c4 P 5f34776fb824418b99b2d9889300b449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:17:27.501446 31963 raft_consensus.cc:2229] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:17:27.502264 31963 raft_consensus.cc:2258] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:17:27.503527 31967 ts_tablet_manager.cc:1905] T 37ae5c73e3814dd89dda6c658c8f77c4 P 5f34776fb824418b99b2d9889300b449: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:17:27.508540 31963 ts_tablet_manager.cc:1905] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:17:27.516185 31963 ts_tablet_manager.cc:1918] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:17:27.516693 31962 tablet_replica.cc:330] T 37ae5c73e3814dd89dda6c658c8f77c4 P 6f753bec541d42d9bcf7da08421aae48: stopping tablet replica
I20260809 12:17:27.516950 31962 raft_consensus.cc:2229] T 37ae5c73e3814dd89dda6c658c8f77c4 P 6f753bec541d42d9bcf7da08421aae48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:17:27.517279 31962 raft_consensus.cc:2258] T 37ae5c73e3814dd89dda6c658c8f77c4 P 6f753bec541d42d9bcf7da08421aae48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:17:27.517604 31963 log.cc:1196] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/37ae5c73e3814dd89dda6c658c8f77c4
I20260809 12:17:27.517913 31967 ts_tablet_manager.cc:1918] T 37ae5c73e3814dd89dda6c658c8f77c4 P 5f34776fb824418b99b2d9889300b449: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:17:27.518059 31967 log.cc:1196] T 37ae5c73e3814dd89dda6c658c8f77c4 P 5f34776fb824418b99b2d9889300b449: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/37ae5c73e3814dd89dda6c658c8f77c4
I20260809 12:17:27.518368 31963 ts_tablet_manager.cc:1939] T 37ae5c73e3814dd89dda6c658c8f77c4 P 918856a3d1054b24b5c3410deca11388: Deleting consensus metadata
I20260809 12:17:27.518548 31967 ts_tablet_manager.cc:1939] T 37ae5c73e3814dd89dda6c658c8f77c4 P 5f34776fb824418b99b2d9889300b449: Deleting consensus metadata
I20260809 12:17:27.518617 31962 ts_tablet_manager.cc:1905] T 37ae5c73e3814dd89dda6c658c8f77c4 P 6f753bec541d42d9bcf7da08421aae48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:17:27.519752 31073 catalog_manager.cc:4841] TS 918856a3d1054b24b5c3410deca11388 (127.19.183.194:43999): tablet 37ae5c73e3814dd89dda6c658c8f77c4 (table test-table [id=29dd438d44154177b2132067c70f45c2]) successfully deleted
I20260809 12:17:27.524281 31962 ts_tablet_manager.cc:1918] T 37ae5c73e3814dd89dda6c658c8f77c4 P 6f753bec541d42d9bcf7da08421aae48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:17:27.524438 31962 log.cc:1196] T 37ae5c73e3814dd89dda6c658c8f77c4 P 6f753bec541d42d9bcf7da08421aae48: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/37ae5c73e3814dd89dda6c658c8f77c4
I20260809 12:17:27.524942 31962 ts_tablet_manager.cc:1939] T 37ae5c73e3814dd89dda6c658c8f77c4 P 6f753bec541d42d9bcf7da08421aae48: Deleting consensus metadata
I20260809 12:17:27.525941 31076 catalog_manager.cc:4841] TS 5f34776fb824418b99b2d9889300b449 (127.19.183.195:35519): tablet 37ae5c73e3814dd89dda6c658c8f77c4 (table test-table [id=29dd438d44154177b2132067c70f45c2]) successfully deleted
I20260809 12:17:27.526511 31072 catalog_manager.cc:4841] TS 6f753bec541d42d9bcf7da08421aae48 (127.19.183.193:42711): tablet 37ae5c73e3814dd89dda6c658c8f77c4 (table test-table [id=29dd438d44154177b2132067c70f45c2]) successfully deleted
Aug 09 12:17:27 dist-test-slave-39l8 krb5kdc[31025](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
W20260809 12:17:27.572219 20191 mini_kdc.cc:317] Time spent kinit for test-admin: real 0.104s user 0.000s sys 0.023s
Aug 09 12:17:27 dist-test-slave-39l8 krb5kdc[31025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:27 dist-test-slave-39l8 krb5kdc[31025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 12:17:27.588645 31919 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:27.575663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53176 (local address 127.19.183.254:43903)
0809 12:17:27.575881 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:27.575890 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:27.576300 (+ 410us) server_negotiation.cc:407] Connection header received
0809 12:17:27.576480 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:27.576486 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:27.576577 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:27.576697 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:27.578400 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.579565 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.581494 (+ 1929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.582178 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.582312 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:27.588363 (+ 6051us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53176: BlockingRecv error: recv got EOF from 127.0.0.1:53176 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 12:17:27 dist-test-slave-39l8 krb5kdc[31025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM
I20260809 12:17:27.612025 31919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:27.593138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53188 (local address 127.19.183.254:43903)
0809 12:17:27.593323 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:27.593330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:27.594280 (+ 950us) server_negotiation.cc:407] Connection header received
0809 12:17:27.594455 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:27.594461 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:27.594552 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:27.594666 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:27.596167 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.597278 (+ 1111us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.599134 (+ 1856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.599796 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.599922 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:27.605169 (+ 5247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:27.605176 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:27.605180 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:27.605243 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:27.608812 (+ 3569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:27.609665 (+ 853us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:27.609671 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:27.609677 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:27.609862 (+ 185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:27.610309 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:27.610316 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:27.610320 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:27.610697 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:27.611033 (+ 336us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:27.611378 (+ 345us) server_negotiation.cc:299] Negotiation successful
0809 12:17:27.611647 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 12:17:27.616168 31091 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:17:27.616945 31091 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:17:27.625460 31514 tablet_service.cc:1467] Processing CreateTablet for tablet c883c40d89c64e9a84dd587153519930 (DEFAULT_TABLE table=test-table [id=0a48f4e082df4aa3886705906b2849fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:17:27.625505 31288 tablet_service.cc:1467] Processing CreateTablet for tablet c883c40d89c64e9a84dd587153519930 (DEFAULT_TABLE table=test-table [id=0a48f4e082df4aa3886705906b2849fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:17:27.625945 31514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c883c40d89c64e9a84dd587153519930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:27.626011 31288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c883c40d89c64e9a84dd587153519930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:27.629279 31717 tablet_service.cc:1467] Processing CreateTablet for tablet c883c40d89c64e9a84dd587153519930 (DEFAULT_TABLE table=test-table [id=0a48f4e082df4aa3886705906b2849fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:17:27.629730 31717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c883c40d89c64e9a84dd587153519930. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:27.633198 31865 tablet_bootstrap.cc:492] T c883c40d89c64e9a84dd587153519930 P 918856a3d1054b24b5c3410deca11388: Bootstrap starting.
I20260809 12:17:27.634938 31865 tablet_bootstrap.cc:654] T c883c40d89c64e9a84dd587153519930 P 918856a3d1054b24b5c3410deca11388: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:27.643929 31864 tablet_bootstrap.cc:492] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48: Bootstrap starting.
I20260809 12:17:27.645531 31864 tablet_bootstrap.cc:654] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:27.646762 31865 tablet_bootstrap.cc:492] T c883c40d89c64e9a84dd587153519930 P 918856a3d1054b24b5c3410deca11388: No bootstrap required, opened a new log
I20260809 12:17:27.646919 31865 ts_tablet_manager.cc:1397] T c883c40d89c64e9a84dd587153519930 P 918856a3d1054b24b5c3410deca11388: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.012s
I20260809 12:17:27.647706 31865 raft_consensus.cc:348] T c883c40d89c64e9a84dd587153519930 P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.647967 31865 raft_consensus.cc:374] T c883c40d89c64e9a84dd587153519930 P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:27.648077 31865 raft_consensus.cc:729] T c883c40d89c64e9a84dd587153519930 P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 918856a3d1054b24b5c3410deca11388, State: Initialized, Role: FOLLOWER
I20260809 12:17:27.648375 31865 consensus_queue.cc:260] T c883c40d89c64e9a84dd587153519930 P 918856a3d1054b24b5c3410deca11388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.651494 31865 ts_tablet_manager.cc:1428] T c883c40d89c64e9a84dd587153519930 P 918856a3d1054b24b5c3410deca11388: Time spent starting tablet: real 0.004s user 0.000s sys 0.002s
I20260809 12:17:27.651605 31867 tablet_bootstrap.cc:492] T c883c40d89c64e9a84dd587153519930 P 5f34776fb824418b99b2d9889300b449: Bootstrap starting.
I20260809 12:17:27.653455 31867 tablet_bootstrap.cc:654] T c883c40d89c64e9a84dd587153519930 P 5f34776fb824418b99b2d9889300b449: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:27.658167 31867 tablet_bootstrap.cc:492] T c883c40d89c64e9a84dd587153519930 P 5f34776fb824418b99b2d9889300b449: No bootstrap required, opened a new log
I20260809 12:17:27.658321 31867 ts_tablet_manager.cc:1397] T c883c40d89c64e9a84dd587153519930 P 5f34776fb824418b99b2d9889300b449: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.000s
I20260809 12:17:27.659168 31867 raft_consensus.cc:348] T c883c40d89c64e9a84dd587153519930 P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.659437 31867 raft_consensus.cc:374] T c883c40d89c64e9a84dd587153519930 P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:27.659552 31867 raft_consensus.cc:729] T c883c40d89c64e9a84dd587153519930 P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f34776fb824418b99b2d9889300b449, State: Initialized, Role: FOLLOWER
I20260809 12:17:27.660118 31867 consensus_queue.cc:260] T c883c40d89c64e9a84dd587153519930 P 5f34776fb824418b99b2d9889300b449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.660853 31867 ts_tablet_manager.cc:1428] T c883c40d89c64e9a84dd587153519930 P 5f34776fb824418b99b2d9889300b449: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 12:17:27.663187 31864 tablet_bootstrap.cc:492] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48: No bootstrap required, opened a new log
I20260809 12:17:27.663400 31864 ts_tablet_manager.cc:1397] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48: Time spent bootstrapping tablet: real 0.020s user 0.003s sys 0.007s
I20260809 12:17:27.664270 31864 raft_consensus.cc:348] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.664556 31864 raft_consensus.cc:374] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:27.664665 31864 raft_consensus.cc:729] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f753bec541d42d9bcf7da08421aae48, State: Initialized, Role: FOLLOWER
I20260809 12:17:27.664898 31864 consensus_queue.cc:260] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.665656 31864 ts_tablet_manager.cc:1428] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 12:17:27.706373 31891 raft_consensus.cc:482] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:17:27.706598 31891 raft_consensus.cc:504] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.707398 31891 leader_election.cc:288] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 918856a3d1054b24b5c3410deca11388 (127.19.183.194:43999), 5f34776fb824418b99b2d9889300b449 (127.19.183.195:35519)
I20260809 12:17:27.708011 31553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c883c40d89c64e9a84dd587153519930" candidate_uuid: "6f753bec541d42d9bcf7da08421aae48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918856a3d1054b24b5c3410deca11388" is_pre_election: true
I20260809 12:17:27.708302 31553 raft_consensus.cc:2454] T c883c40d89c64e9a84dd587153519930 P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f753bec541d42d9bcf7da08421aae48 in term 0.
I20260809 12:17:27.708226 31748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c883c40d89c64e9a84dd587153519930" candidate_uuid: "6f753bec541d42d9bcf7da08421aae48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f34776fb824418b99b2d9889300b449" is_pre_election: true
I20260809 12:17:27.708525 31748 raft_consensus.cc:2454] T c883c40d89c64e9a84dd587153519930 P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f753bec541d42d9bcf7da08421aae48 in term 0.
I20260809 12:17:27.708848 31218 leader_election.cc:302] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f753bec541d42d9bcf7da08421aae48, 918856a3d1054b24b5c3410deca11388; no voters:
I20260809 12:17:27.709312 31891 raft_consensus.cc:2781] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:17:27.709441 31891 raft_consensus.cc:482] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:17:27.709558 31891 raft_consensus.cc:3037] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:27.712785 31891 raft_consensus.cc:504] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.713858 31553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c883c40d89c64e9a84dd587153519930" candidate_uuid: "6f753bec541d42d9bcf7da08421aae48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918856a3d1054b24b5c3410deca11388"
I20260809 12:17:27.713903 31748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c883c40d89c64e9a84dd587153519930" candidate_uuid: "6f753bec541d42d9bcf7da08421aae48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f34776fb824418b99b2d9889300b449"
I20260809 12:17:27.714080 31553 raft_consensus.cc:3037] T c883c40d89c64e9a84dd587153519930 P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:27.714109 31748 raft_consensus.cc:3037] T c883c40d89c64e9a84dd587153519930 P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:27.717304 31553 raft_consensus.cc:2454] T c883c40d89c64e9a84dd587153519930 P 918856a3d1054b24b5c3410deca11388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f753bec541d42d9bcf7da08421aae48 in term 1.
I20260809 12:17:27.717403 31748 raft_consensus.cc:2454] T c883c40d89c64e9a84dd587153519930 P 5f34776fb824418b99b2d9889300b449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f753bec541d42d9bcf7da08421aae48 in term 1.
I20260809 12:17:27.717890 31219 leader_election.cc:302] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f34776fb824418b99b2d9889300b449, 6f753bec541d42d9bcf7da08421aae48; no voters:
I20260809 12:17:27.718312 31891 leader_election.cc:288] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48 [CANDIDATE]: Term 1 election: Requested vote from peers 918856a3d1054b24b5c3410deca11388 (127.19.183.194:43999), 5f34776fb824418b99b2d9889300b449 (127.19.183.195:35519)
I20260809 12:17:27.718561 31891 raft_consensus.cc:2781] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:17:27.718767 31891 raft_consensus.cc:686] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48 [term 1 LEADER]: Becoming Leader. State: Replica: 6f753bec541d42d9bcf7da08421aae48, State: Running, Role: LEADER
I20260809 12:17:27.719321 31891 consensus_queue.cc:237] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.731627 31091 catalog_manager.cc:5495] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f753bec541d42d9bcf7da08421aae48 (127.19.183.193). New cstate: current_term: 1 leader_uuid: "6f753bec541d42d9bcf7da08421aae48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } health_report { overall_health: UNKNOWN } } }
I20260809 12:17:27.769052 31992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:27.752706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43422 (local address 127.19.183.193:42711)
0809 12:17:27.762436 (+ 9730us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:27.762442 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:27.762462 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:17:27.762538 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:27.762543 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:27.762627 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:27.762767 (+ 140us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:17:27.763234 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.764294 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.766001 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:27.766658 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:27.767361 (+ 703us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:27.767463 (+ 102us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:17:27.767585 (+ 122us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:17:27.768585 (+ 1000us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:27.768639 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 12:17:27.768733 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9575,"thread_start_us":9422,"threads_started":1}
I20260809 12:17:27.773109 31748 raft_consensus.cc:1261] T c883c40d89c64e9a84dd587153519930 P 5f34776fb824418b99b2d9889300b449 [term 1 FOLLOWER]: Refusing update from remote peer 6f753bec541d42d9bcf7da08421aae48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:17:27.773777 31553 raft_consensus.cc:1261] T c883c40d89c64e9a84dd587153519930 P 918856a3d1054b24b5c3410deca11388 [term 1 FOLLOWER]: Refusing update from remote peer 6f753bec541d42d9bcf7da08421aae48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:17:27.774564 31987 consensus_queue.cc:1035] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:17:27.775920 31987 consensus_queue.cc:1035] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:17:27.808245 31091 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:17:27.808522 31091 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:17:27.813915 31091 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112: Sending DeleteTablet for 3 replicas of tablet c883c40d89c64e9a84dd587153519930
I20260809 12:17:27.814656 31288 tablet_service.cc:1514] Processing DeleteTablet for tablet c883c40d89c64e9a84dd587153519930 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:17:27 UTC) from {username='oryx', principal='oryx/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:43402
I20260809 12:17:27.815065 31514 tablet_service.cc:1514] Processing DeleteTablet for tablet c883c40d89c64e9a84dd587153519930 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:17:27 UTC) from {username='oryx', principal='oryx/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:34530
I20260809 12:17:27.815119 31962 tablet_replica.cc:330] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48: stopping tablet replica
I20260809 12:17:27.815372 31963 tablet_replica.cc:330] T c883c40d89c64e9a84dd587153519930 P 918856a3d1054b24b5c3410deca11388: stopping tablet replica
I20260809 12:17:27.815373 31962 raft_consensus.cc:2229] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:17:27.815637 31963 raft_consensus.cc:2229] T c883c40d89c64e9a84dd587153519930 P 918856a3d1054b24b5c3410deca11388 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:17:27.815703 31962 raft_consensus.cc:2258] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:17:27.815892 31963 raft_consensus.cc:2258] T c883c40d89c64e9a84dd587153519930 P 918856a3d1054b24b5c3410deca11388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:17:27.816068 31717 tablet_service.cc:1514] Processing DeleteTablet for tablet c883c40d89c64e9a84dd587153519930 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:17:27 UTC) from {username='oryx', principal='oryx/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:50392
I20260809 12:17:27.816546 31967 tablet_replica.cc:330] T c883c40d89c64e9a84dd587153519930 P 5f34776fb824418b99b2d9889300b449: stopping tablet replica
I20260809 12:17:27.816754 31967 raft_consensus.cc:2229] T c883c40d89c64e9a84dd587153519930 P 5f34776fb824418b99b2d9889300b449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:17:27.816876 31963 ts_tablet_manager.cc:1905] T c883c40d89c64e9a84dd587153519930 P 918856a3d1054b24b5c3410deca11388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:17:27.817022 31967 raft_consensus.cc:2258] T c883c40d89c64e9a84dd587153519930 P 5f34776fb824418b99b2d9889300b449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:17:27.818382 31962 ts_tablet_manager.cc:1905] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:17:27.822122 31963 ts_tablet_manager.cc:1918] T c883c40d89c64e9a84dd587153519930 P 918856a3d1054b24b5c3410deca11388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:17:27.822309 31963 log.cc:1196] T c883c40d89c64e9a84dd587153519930 P 918856a3d1054b24b5c3410deca11388: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/c883c40d89c64e9a84dd587153519930
I20260809 12:17:27.822794 31963 ts_tablet_manager.cc:1939] T c883c40d89c64e9a84dd587153519930 P 918856a3d1054b24b5c3410deca11388: Deleting consensus metadata
I20260809 12:17:27.823509 31967 ts_tablet_manager.cc:1905] T c883c40d89c64e9a84dd587153519930 P 5f34776fb824418b99b2d9889300b449: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:17:27.823854 31073 catalog_manager.cc:4841] TS 918856a3d1054b24b5c3410deca11388 (127.19.183.194:43999): tablet c883c40d89c64e9a84dd587153519930 (table test-table [id=0a48f4e082df4aa3886705906b2849fe]) successfully deleted
I20260809 12:17:27.828760 31962 ts_tablet_manager.cc:1918] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:17:27.828917 31962 log.cc:1196] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/c883c40d89c64e9a84dd587153519930
I20260809 12:17:27.829312 31962 ts_tablet_manager.cc:1939] T c883c40d89c64e9a84dd587153519930 P 6f753bec541d42d9bcf7da08421aae48: Deleting consensus metadata
I20260809 12:17:27.829950 31967 ts_tablet_manager.cc:1918] T c883c40d89c64e9a84dd587153519930 P 5f34776fb824418b99b2d9889300b449: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:17:27.830112 31967 log.cc:1196] T c883c40d89c64e9a84dd587153519930 P 5f34776fb824418b99b2d9889300b449: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/c883c40d89c64e9a84dd587153519930
I20260809 12:17:27.830188 31072 catalog_manager.cc:4841] TS 6f753bec541d42d9bcf7da08421aae48 (127.19.183.193:42711): tablet c883c40d89c64e9a84dd587153519930 (table test-table [id=0a48f4e082df4aa3886705906b2849fe]) successfully deleted
I20260809 12:17:27.830250 31091 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 12:17:27.830506 31967 ts_tablet_manager.cc:1939] T c883c40d89c64e9a84dd587153519930 P 5f34776fb824418b99b2d9889300b449: Deleting consensus metadata
W20260809 12:17:27.830951 31091 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:17:27.831796 31076 catalog_manager.cc:4841] TS 5f34776fb824418b99b2d9889300b449 (127.19.183.195:35519): tablet c883c40d89c64e9a84dd587153519930 (table test-table [id=0a48f4e082df4aa3886705906b2849fe]) successfully deleted
I20260809 12:17:27.838020 31288 tablet_service.cc:1467] Processing CreateTablet for tablet 185aa94834be4bc5a1e7eb07c0f9ed4d (DEFAULT_TABLE table=test-table [id=2744aa87921c437086fb490e73de27b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:17:27.838433 31288 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 185aa94834be4bc5a1e7eb07c0f9ed4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:27.838431 31514 tablet_service.cc:1467] Processing CreateTablet for tablet 185aa94834be4bc5a1e7eb07c0f9ed4d (DEFAULT_TABLE table=test-table [id=2744aa87921c437086fb490e73de27b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:17:27.839306 31717 tablet_service.cc:1467] Processing CreateTablet for tablet 185aa94834be4bc5a1e7eb07c0f9ed4d (DEFAULT_TABLE table=test-table [id=2744aa87921c437086fb490e73de27b7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:17:27.839628 31514 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 185aa94834be4bc5a1e7eb07c0f9ed4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:27.839744 31717 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 185aa94834be4bc5a1e7eb07c0f9ed4d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:27.844705 31864 tablet_bootstrap.cc:492] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48: Bootstrap starting.
I20260809 12:17:27.846184 31867 tablet_bootstrap.cc:492] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 5f34776fb824418b99b2d9889300b449: Bootstrap starting.
I20260809 12:17:27.846311 31864 tablet_bootstrap.cc:654] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:27.847774 31867 tablet_bootstrap.cc:654] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 5f34776fb824418b99b2d9889300b449: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:27.848873 31865 tablet_bootstrap.cc:492] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 918856a3d1054b24b5c3410deca11388: Bootstrap starting.
I20260809 12:17:27.850611 31865 tablet_bootstrap.cc:654] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 918856a3d1054b24b5c3410deca11388: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:27.856683 31865 tablet_bootstrap.cc:492] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 918856a3d1054b24b5c3410deca11388: No bootstrap required, opened a new log
I20260809 12:17:27.856863 31865 ts_tablet_manager.cc:1397] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 918856a3d1054b24b5c3410deca11388: Time spent bootstrapping tablet: real 0.008s user 0.004s sys 0.001s
I20260809 12:17:27.857668 31865 raft_consensus.cc:348] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.857877 31864 tablet_bootstrap.cc:492] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48: No bootstrap required, opened a new log
I20260809 12:17:27.857919 31865 raft_consensus.cc:374] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:27.858071 31864 ts_tablet_manager.cc:1397] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48: Time spent bootstrapping tablet: real 0.013s user 0.003s sys 0.009s
I20260809 12:17:27.858089 31865 raft_consensus.cc:729] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 918856a3d1054b24b5c3410deca11388, State: Initialized, Role: FOLLOWER
I20260809 12:17:27.858382 31865 consensus_queue.cc:260] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 918856a3d1054b24b5c3410deca11388 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.858922 31864 raft_consensus.cc:348] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.859130 31864 raft_consensus.cc:374] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:27.859216 31864 raft_consensus.cc:729] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f753bec541d42d9bcf7da08421aae48, State: Initialized, Role: FOLLOWER
I20260809 12:17:27.859261 31867 tablet_bootstrap.cc:492] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 5f34776fb824418b99b2d9889300b449: No bootstrap required, opened a new log
I20260809 12:17:27.859445 31867 ts_tablet_manager.cc:1397] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 5f34776fb824418b99b2d9889300b449: Time spent bootstrapping tablet: real 0.013s user 0.002s sys 0.003s
I20260809 12:17:27.859537 31864 consensus_queue.cc:260] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.860044 31864 ts_tablet_manager.cc:1428] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 12:17:27.860795 31867 raft_consensus.cc:348] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.861089 31867 raft_consensus.cc:374] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:27.861183 31867 raft_consensus.cc:729] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f34776fb824418b99b2d9889300b449, State: Initialized, Role: FOLLOWER
I20260809 12:17:27.861455 31867 consensus_queue.cc:260] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 5f34776fb824418b99b2d9889300b449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:27.864563 31865 ts_tablet_manager.cc:1428] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 918856a3d1054b24b5c3410deca11388: Time spent starting tablet: real 0.008s user 0.001s sys 0.001s
I20260809 12:17:27.865003 31867 ts_tablet_manager.cc:1428] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 5f34776fb824418b99b2d9889300b449: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260809 12:17:28.034077 31987 raft_consensus.cc:482] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:17:28.034281 31987 raft_consensus.cc:504] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:28.035028 31987 leader_election.cc:288] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 918856a3d1054b24b5c3410deca11388 (127.19.183.194:43999), 5f34776fb824418b99b2d9889300b449 (127.19.183.195:35519)
I20260809 12:17:28.035533 31553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "185aa94834be4bc5a1e7eb07c0f9ed4d" candidate_uuid: "6f753bec541d42d9bcf7da08421aae48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918856a3d1054b24b5c3410deca11388" is_pre_election: true
I20260809 12:17:28.035621 31748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "185aa94834be4bc5a1e7eb07c0f9ed4d" candidate_uuid: "6f753bec541d42d9bcf7da08421aae48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f34776fb824418b99b2d9889300b449" is_pre_election: true
I20260809 12:17:28.035777 31553 raft_consensus.cc:2454] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f753bec541d42d9bcf7da08421aae48 in term 0.
I20260809 12:17:28.035907 31748 raft_consensus.cc:2454] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6f753bec541d42d9bcf7da08421aae48 in term 0.
I20260809 12:17:28.036283 31218 leader_election.cc:302] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6f753bec541d42d9bcf7da08421aae48, 918856a3d1054b24b5c3410deca11388; no voters:
I20260809 12:17:28.036562 31987 raft_consensus.cc:2781] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:17:28.036679 31987 raft_consensus.cc:482] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:17:28.036783 31987 raft_consensus.cc:3037] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:28.038841 31987 raft_consensus.cc:504] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:28.039458 31987 leader_election.cc:288] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48 [CANDIDATE]: Term 1 election: Requested vote from peers 918856a3d1054b24b5c3410deca11388 (127.19.183.194:43999), 5f34776fb824418b99b2d9889300b449 (127.19.183.195:35519)
I20260809 12:17:28.039858 31553 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "185aa94834be4bc5a1e7eb07c0f9ed4d" candidate_uuid: "6f753bec541d42d9bcf7da08421aae48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "918856a3d1054b24b5c3410deca11388"
I20260809 12:17:28.039899 31748 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "185aa94834be4bc5a1e7eb07c0f9ed4d" candidate_uuid: "6f753bec541d42d9bcf7da08421aae48" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f34776fb824418b99b2d9889300b449"
I20260809 12:17:28.040045 31553 raft_consensus.cc:3037] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 918856a3d1054b24b5c3410deca11388 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:28.040081 31748 raft_consensus.cc:3037] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 5f34776fb824418b99b2d9889300b449 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:28.042218 31748 raft_consensus.cc:2454] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 5f34776fb824418b99b2d9889300b449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f753bec541d42d9bcf7da08421aae48 in term 1.
I20260809 12:17:28.042508 31553 raft_consensus.cc:2454] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 918856a3d1054b24b5c3410deca11388 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6f753bec541d42d9bcf7da08421aae48 in term 1.
I20260809 12:17:28.042631 31219 leader_election.cc:302] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f34776fb824418b99b2d9889300b449, 6f753bec541d42d9bcf7da08421aae48; no voters:
I20260809 12:17:28.042955 31987 raft_consensus.cc:2781] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:17:28.043128 31987 raft_consensus.cc:686] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48 [term 1 LEADER]: Becoming Leader. State: Replica: 6f753bec541d42d9bcf7da08421aae48, State: Running, Role: LEADER
I20260809 12:17:28.043407 31987 consensus_queue.cc:237] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } }
I20260809 12:17:28.045756 31090 catalog_manager.cc:5495] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6f753bec541d42d9bcf7da08421aae48 (127.19.183.193). New cstate: current_term: 1 leader_uuid: "6f753bec541d42d9bcf7da08421aae48" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6f753bec541d42d9bcf7da08421aae48" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 42711 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 } health_report { overall_health: UNKNOWN } } }
I20260809 12:17:28.098451 32011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:28.093555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:49609 (local address 127.19.183.195:35519)
0809 12:17:28.094059 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:28.094065 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:28.094310 (+ 245us) server_negotiation.cc:407] Connection header received
0809 12:17:28.094450 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:28.094453 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:28.094515 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:28.094615 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:17:28.095142 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:28.095784 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:28.097036 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:28.097432 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:28.097517 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:28.097896 (+ 379us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:17:28.097984 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:17:28.098046 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:28.098182 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 12:17:28.098238 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":254,"threads_started":1}
I20260809 12:17:28.098495 32013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:28.093562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:35519 (local address 127.19.183.193:49609)
0809 12:17:28.094211 (+ 649us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:28.094227 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:28.094355 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:28.094657 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:28.094660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:28.094669 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:17:28.094992 (+ 323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:28.095000 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:28.095901 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:28.095904 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:28.096925 (+ 1021us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:28.096936 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:28.097570 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:28.097574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:28.097705 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:28.097753 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:17:28.098067 (+ 314us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:17:28.098075 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 12:17:28.098132 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 12:17:28.098221 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"thread_start_us":328,"threads_started":1}
I20260809 12:17:28.105752 31553 raft_consensus.cc:1261] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 918856a3d1054b24b5c3410deca11388 [term 1 FOLLOWER]: Refusing update from remote peer 6f753bec541d42d9bcf7da08421aae48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:17:28.105886 31748 raft_consensus.cc:1261] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 5f34776fb824418b99b2d9889300b449 [term 1 FOLLOWER]: Refusing update from remote peer 6f753bec541d42d9bcf7da08421aae48: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:17:28.106361 31996 consensus_queue.cc:1035] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "918856a3d1054b24b5c3410deca11388" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 43999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:17:28.106663 31987 consensus_queue.cc:1035] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f34776fb824418b99b2d9889300b449" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35519 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:17:28.113533 31886 mvcc.cc:205] Tried to move back new op lower bound from 7316594065836175360 to 7316594065587785728. Current Snapshot: MvccSnapshot[applied={T|T < 7316594065836175360}]
I20260809 12:17:28.131735 31992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:28.125469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:57797 (local address 127.19.183.193:42711)
0809 12:17:28.125654 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:28.125661 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:28.126161 (+ 500us) server_negotiation.cc:407] Connection header received
0809 12:17:28.126354 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:28.126359 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:28.126448 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:28.126545 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:17:28.127022 (+ 477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:28.127762 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:28.129536 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:28.130149 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:28.130222 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:28.130886 (+ 664us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:17:28.131056 (+ 170us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:17:28.131148 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:28.131351 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 12:17:28.131467 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 12:17:28.132292 32020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:28.125409 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:42711 (local address 127.19.183.195:57797)
0809 12:17:28.126057 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:28.126072 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:28.126225 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:28.126587 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:28.126610 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:28.126623 (+ 13us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 12:17:28.126895 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:28.126904 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:28.127931 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:28.127936 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:28.129371 (+ 1435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:28.129384 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:28.130401 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:28.130405 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:28.130565 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:28.130604 (+ 39us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 12:17:28.131208 (+ 604us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 12:17:28.131220 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 12:17:28.131304 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 12:17:28.131438 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":346,"threads_started":1}
I20260809 12:17:28.165870 31090 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:17:28.166128 31090 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:17:28.168851 31090 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ddb76241bc9a4ecdbb72c7b7bbe9a112: Sending DeleteTablet for 3 replicas of tablet 185aa94834be4bc5a1e7eb07c0f9ed4d
I20260809 12:17:28.169660 31514 tablet_service.cc:1514] Processing DeleteTablet for tablet 185aa94834be4bc5a1e7eb07c0f9ed4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:17:28 UTC) from {username='oryx', principal='oryx/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:34530
I20260809 12:17:28.169673 31288 tablet_service.cc:1514] Processing DeleteTablet for tablet 185aa94834be4bc5a1e7eb07c0f9ed4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:17:28 UTC) from {username='oryx', principal='oryx/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:43402
I20260809 12:17:28.169982 31962 tablet_replica.cc:330] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48: stopping tablet replica
I20260809 12:17:28.169974 31717 tablet_service.cc:1514] Processing DeleteTablet for tablet 185aa94834be4bc5a1e7eb07c0f9ed4d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:17:28 UTC) from {username='oryx', principal='oryx/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:50392
I20260809 12:17:28.170190 31962 raft_consensus.cc:2229] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:17:28.170246 31967 tablet_replica.cc:330] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 5f34776fb824418b99b2d9889300b449: stopping tablet replica
I20260809 12:17:28.170430 31967 raft_consensus.cc:2229] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 5f34776fb824418b99b2d9889300b449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:17:28.170517 31962 raft_consensus.cc:2258] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:17:28.170663 31967 raft_consensus.cc:2258] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 5f34776fb824418b99b2d9889300b449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:17:28.170841 31963 tablet_replica.cc:330] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 918856a3d1054b24b5c3410deca11388: stopping tablet replica
I20260809 12:17:28.170921 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 31191
I20260809 12:17:28.171037 31963 raft_consensus.cc:2229] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 918856a3d1054b24b5c3410deca11388 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:17:28.171270 31963 raft_consensus.cc:2258] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 918856a3d1054b24b5c3410deca11388 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:17:28.172153 31967 ts_tablet_manager.cc:1905] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 5f34776fb824418b99b2d9889300b449: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:17:28.172323 31962 ts_tablet_manager.cc:1905] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:17:28.173487 31963 ts_tablet_manager.cc:1905] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 918856a3d1054b24b5c3410deca11388: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:17:28.179023 31967 ts_tablet_manager.cc:1918] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 5f34776fb824418b99b2d9889300b449: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:17:28.179171 31967 log.cc:1196] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 5f34776fb824418b99b2d9889300b449: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/185aa94834be4bc5a1e7eb07c0f9ed4d
I20260809 12:17:28.179260 31962 ts_tablet_manager.cc:1918] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:17:28.179404 31962 log.cc:1196] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/185aa94834be4bc5a1e7eb07c0f9ed4d
I20260809 12:17:28.179664 31967 ts_tablet_manager.cc:1939] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 5f34776fb824418b99b2d9889300b449: Deleting consensus metadata
I20260809 12:17:28.179982 31962 ts_tablet_manager.cc:1939] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 6f753bec541d42d9bcf7da08421aae48: Deleting consensus metadata
I20260809 12:17:28.180828 31076 catalog_manager.cc:4841] TS 5f34776fb824418b99b2d9889300b449 (127.19.183.195:35519): tablet 185aa94834be4bc5a1e7eb07c0f9ed4d (table test-table [id=2744aa87921c437086fb490e73de27b7]) successfully deleted
I20260809 12:17:28.180964 31072 catalog_manager.cc:4841] TS 6f753bec541d42d9bcf7da08421aae48 (127.19.183.193:42711): tablet 185aa94834be4bc5a1e7eb07c0f9ed4d (table test-table [id=2744aa87921c437086fb490e73de27b7]) successfully deleted
Aug 09 12:17:28 dist-test-slave-39l8 krb5kdc[31025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.183.193@KRBTEST.COM
I20260809 12:17:28.185873 31963 ts_tablet_manager.cc:1918] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 918856a3d1054b24b5c3410deca11388: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 12:17:28.186019 31963 log.cc:1196] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 918856a3d1054b24b5c3410deca11388: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/185aa94834be4bc5a1e7eb07c0f9ed4d
I20260809 12:17:28.186470 31963 ts_tablet_manager.cc:1939] T 185aa94834be4bc5a1e7eb07c0f9ed4d P 918856a3d1054b24b5c3410deca11388: Deleting consensus metadata
I20260809 12:17:28.187680 31073 catalog_manager.cc:4841] TS 918856a3d1054b24b5c3410deca11388 (127.19.183.194:43999): tablet 185aa94834be4bc5a1e7eb07c0f9ed4d (table test-table [id=2744aa87921c437086fb490e73de27b7]) successfully deleted
I20260809 12:17:28.193240 31992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:28.172566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43436 (local address 127.19.183.193:42711)
0809 12:17:28.173949 (+ 1383us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:28.173960 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:28.173977 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:17:28.174050 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:28.174056 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:28.174141 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:28.174236 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:28.175995 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:28.177247 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:28.179271 (+ 2024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:28.180081 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:28.180196 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:28.185956 (+ 5760us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:28.185976 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:28.185999 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:28.186059 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:28.189823 (+ 3764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:28.190906 (+ 1083us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:28.190916 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:28.190919 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:28.191188 (+ 269us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:28.191586 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:28.191592 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:28.191595 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:28.192007 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:28.192336 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:28.192670 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 12:17:28.192901 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1263}
I20260809 12:17:28.193987 31403 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:28.328900 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 31191
I20260809 12:17:28.361680 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 31414
Aug 09 12:17:28 dist-test-slave-39l8 krb5kdc[31025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.183.194@KRBTEST.COM
I20260809 12:17:28.376184 32025 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:28.362510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34546 (local address 127.19.183.194:43999)
0809 12:17:28.363058 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:28.363063 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:28.363080 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:17:28.363136 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:28.363140 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:28.363224 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:28.363321 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:28.364588 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:28.365387 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:28.366582 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:28.367112 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:28.367209 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:28.371208 (+ 3999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:28.371222 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:28.371237 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:28.371267 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:28.373832 (+ 2565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:28.374352 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:28.374358 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:28.374360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:28.374476 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:28.374898 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:28.374901 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:28.374903 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:28.375217 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:28.375461 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:28.375780 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 12:17:28.375929 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":432,"thread_start_us":302,"threads_started":1}
I20260809 12:17:28.376842 31609 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:28.501777 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 31414
I20260809 12:17:28.539151 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 31619
Aug 09 12:17:28 dist-test-slave-39l8 krb5kdc[31025](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277847, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.19.183.195@KRBTEST.COM
I20260809 12:17:28.553664 32011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:28.539918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50402 (local address 127.19.183.195:35519)
0809 12:17:28.540093 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:28.540097 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:28.540131 (+ 34us) server_negotiation.cc:407] Connection header received
0809 12:17:28.540268 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:28.540272 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:28.540334 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:28.540459 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:28.541740 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:28.542440 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:28.543701 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:28.544099 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:28.544191 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:28.548269 (+ 4078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:28.548292 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:28.548316 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:28.548363 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:28.551428 (+ 3065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:28.552026 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:28.552033 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:28.552035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:28.552165 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:28.552480 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:28.552483 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:28.552485 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:28.552821 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:28.553053 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:28.553268 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 12:17:28.553419 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 12:17:28.554318 31823 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:28.677671 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 31619
I20260809 12:17:28.677721 32028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:28.676871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:42711 (local address 127.19.183.195:34491)
0809 12:17:28.677407 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:28.677550 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:42711: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":307,"threads_started":1}
I20260809 12:17:28.708921 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 31045
I20260809 12:17:28.710093 31147 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:28.823542 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 31045
2026-08-09T12:17:28Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (6354 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:17:28 dist-test-slave-39l8 krb5kdc[32034](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:17:28 dist-test-slave-39l8 krb5kdc[32034](info): set up 2 sockets
Aug 09 12:17:28 dist-test-slave-39l8 krb5kdc[32034](info): commencing operation
krb5kdc: starting...
W20260809 12:17:30.116525 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.238s user 0.003s sys 0.030s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:17:30 dist-test-slave-39l8 krb5kdc[32034](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:17:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:17:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:17:30.461527 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:46155
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:38749
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:46155
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:17:31.043326 32048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:31.043825 32048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:31.043982 32048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:31.055413 32048 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:17:31.055560 32048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:31.055672 32048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:31.055776 32048 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:17:31.055867 32048 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:17:31.072196 32048 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:38749
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:46155
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:46155
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:31.074890 32048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:31.077263 32048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:31.097371 32056 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:31.097927 32058 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:31.098009 32054 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:31.099340 32048 server_base.cc:1034] running on GCE node
I20260809 12:17:31.100765 32048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:31.103057 32048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:31.104386 32048 hybrid_clock.cc:648] HybridClock initialized: now 1786277851104299 us; error 107 us; skew 500 ppm
I20260809 12:17:31.104933 32048 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T12:17:31Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (2390 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:17:31 dist-test-slave-39l8 krb5kdc[32065](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:17:31 dist-test-slave-39l8 krb5kdc[32065](info): set up 2 sockets
Aug 09 12:17:31 dist-test-slave-39l8 krb5kdc[32065](info): commencing operation
krb5kdc: starting...
W20260809 12:17:33.416774 20191 mini_kdc.cc:121] Time spent starting KDC: real 2.144s user 0.004s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:17:33 dist-test-slave-39l8 krb5kdc[32065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:17:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:17:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:17:33.763954 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:35275
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:33091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:35275
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:17:34.110968 32082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:34.111325 32082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:34.111487 32082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:34.118665 32082 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:17:34.118830 32082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:34.118944 32082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:34.119055 32082 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:17:34.119135 32082 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:17:34.128612 32082 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:33091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:35275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:35275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:34.129884 32082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:34.131307 32082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:34.143208 32090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:34.143158 32091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:34.143237 32093 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:34.144331 32082 server_base.cc:1034] running on GCE node
I20260809 12:17:34.145128 32082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:34.146538 32082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:34.147781 32082 hybrid_clock.cc:648] HybridClock initialized: now 1786277854147715 us; error 49 us; skew 500 ppm
I20260809 12:17:34.148140 32082 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:34 dist-test-slave-39l8 krb5kdc[32065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277854, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:34.151475 32082 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:17:34.152951 32082 webserver.cc:460] Webserver started at http://127.19.183.254:43301/ using document root <none> and password file <none>
I20260809 12:17:34.153674 32082 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:34.153813 32082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:34.154156 32082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:34.157176 32082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "48df2a99720c4eb597114faec466d5c9"
format_stamp: "Formatted at 2026-08-09 12:17:34 on dist-test-slave-39l8"
server_key: "f6ffd7f52e712a220505f638fef8425f"
server_key_iv: "3f7c5f93f1e8ebb5ca9d0b4eb7f177e0"
server_key_version: "encryptionkey@0"
I20260809 12:17:34.157958 32082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "48df2a99720c4eb597114faec466d5c9"
format_stamp: "Formatted at 2026-08-09 12:17:34 on dist-test-slave-39l8"
server_key: "f6ffd7f52e712a220505f638fef8425f"
server_key_iv: "3f7c5f93f1e8ebb5ca9d0b4eb7f177e0"
server_key_version: "encryptionkey@0"
I20260809 12:17:34.163192 32082 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.002s
I20260809 12:17:34.166553 32103 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:34.167883 32082 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:17:34.168076 32082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "48df2a99720c4eb597114faec466d5c9"
format_stamp: "Formatted at 2026-08-09 12:17:34 on dist-test-slave-39l8"
server_key: "f6ffd7f52e712a220505f638fef8425f"
server_key_iv: "3f7c5f93f1e8ebb5ca9d0b4eb7f177e0"
server_key_version: "encryptionkey@0"
I20260809 12:17:34.168291 32082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:34.190936 32082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:34.194697 32082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:34.195019 32082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:34.213310 32082 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:35275
I20260809 12:17:34.213340 32181 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:35275 every 8 connection(s)
I20260809 12:17:34.214650 32082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:17:34.216984 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 32082
I20260809 12:17:34.217239 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:17:34.217847 20191 external_mini_cluster.cc:1428] Setting key dcd5fddf045b00082f2fdc12d4d26875
I20260809 12:17:34.218468 32182 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:34.233170 32182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9: Bootstrap starting.
Aug 09 12:17:34 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:17:34.236639 32182 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:34.237756 32182 log.cc:824] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9: Log is configured to *not* fsync() on all Append() calls
I20260809 12:17:34.240988 32182 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9: No bootstrap required, opened a new log
I20260809 12:17:34.242020 32185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:34.220001 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51652 (local address 127.19.183.254:35275)
0809 12:17:34.220796 (+ 795us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:34.220807 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:34.220847 (+ 40us) server_negotiation.cc:407] Connection header received
0809 12:17:34.223323 (+ 2476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:34.223345 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:34.223715 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:34.224011 (+ 296us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:34.225885 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:34.227197 (+ 1312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:34.229121 (+ 1924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:34.229800 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:34.229942 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:34.234693 (+ 4751us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:34.234744 (+ 51us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:34.234764 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:34.234814 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:34.237822 (+ 3008us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:34.238650 (+ 828us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:34.238661 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:34.238672 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:34.238950 (+ 278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:34.239407 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:34.239411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:34.239413 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:34.239894 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:34.240126 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:34.240609 (+ 483us) server_negotiation.cc:299] Negotiation successful
0809 12:17:34.240981 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":351,"threads_started":1}
I20260809 12:17:34.248937 32182 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48df2a99720c4eb597114faec466d5c9" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35275 } }
I20260809 12:17:34.249341 32182 raft_consensus.cc:374] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:34.249459 32182 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48df2a99720c4eb597114faec466d5c9, State: Initialized, Role: FOLLOWER
I20260809 12:17:34.250190 32182 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48df2a99720c4eb597114faec466d5c9" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35275 } }
I20260809 12:17:34.250454 32182 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:17:34.250574 32182 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:17:34.250695 32182 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:34.252887 32182 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48df2a99720c4eb597114faec466d5c9" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35275 } }
I20260809 12:17:34.253317 32182 leader_election.cc:302] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48df2a99720c4eb597114faec466d5c9; no voters:
I20260809 12:17:34.254027 32182 leader_election.cc:288] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:17:34.254153 32187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:17:34.255070 32187 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [term 1 LEADER]: Becoming Leader. State: Replica: 48df2a99720c4eb597114faec466d5c9, State: Running, Role: LEADER
I20260809 12:17:34.255564 32187 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48df2a99720c4eb597114faec466d5c9" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35275 } }
I20260809 12:17:34.256278 32182 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:17:34.258606 32191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48df2a99720c4eb597114faec466d5c9. Latest consensus state: current_term: 1 leader_uuid: "48df2a99720c4eb597114faec466d5c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48df2a99720c4eb597114faec466d5c9" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35275 } } }
I20260809 12:17:34.259094 32191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:34.258903 32190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "48df2a99720c4eb597114faec466d5c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48df2a99720c4eb597114faec466d5c9" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35275 } } }
I20260809 12:17:34.259349 32190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:34.261041 32201 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:17:34.266448 32201 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:17:34.275449 32201 catalog_manager.cc:1347] Generated new cluster ID: 534542138b964705aca00b43d5b1a1b9
I20260809 12:17:34.275549 32201 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:17:34.306650 32201 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:17:34.307893 32201 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:17:34.316143 32201 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9: Generated new TSK 0
I20260809 12:17:34.317088 32201 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
I20260809 12:17:34.383991 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35275
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:33091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:17:34.720388 32217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:34.720646 32217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:34.720799 32217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:34.727742 32217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:34.727867 32217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:34.728088 32217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:17:34.737145 32217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:33091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35275
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:34.738292 32217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:34.739550 32217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:34.754834 32227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:34.754920 32228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:34.754967 32231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:34.755928 32217 server_base.cc:1034] running on GCE node
I20260809 12:17:34.756511 32217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:34.757475 32217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:34.758671 32217 hybrid_clock.cc:648] HybridClock initialized: now 1786277854758616 us; error 49 us; skew 500 ppm
I20260809 12:17:34.759009 32217 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:34 dist-test-slave-39l8 krb5kdc[32065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277854, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:34.761987 32217 init.cc:377] Logged in from keytab as kudu/127.19.183.193@KRBTEST.COM (short username kudu)
I20260809 12:17:34.763393 32217 webserver.cc:460] Webserver started at http://127.19.183.193:46315/ using document root <none> and password file <none>
I20260809 12:17:34.763969 32217 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:34.764066 32217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:34.764336 32217 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:34.767539 32217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "2b2eb15d42c24cb98ea5f4ddbcf84e2d"
format_stamp: "Formatted at 2026-08-09 12:17:34 on dist-test-slave-39l8"
server_key: "6055f7c7774da82288d8c27fa624a8a0"
server_key_iv: "8d490ad9aedf88a1da3cce9bc31158d8"
server_key_version: "encryptionkey@0"
I20260809 12:17:34.768246 32217 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "2b2eb15d42c24cb98ea5f4ddbcf84e2d"
format_stamp: "Formatted at 2026-08-09 12:17:34 on dist-test-slave-39l8"
server_key: "6055f7c7774da82288d8c27fa624a8a0"
server_key_iv: "8d490ad9aedf88a1da3cce9bc31158d8"
server_key_version: "encryptionkey@0"
I20260809 12:17:34.773285 32217 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260809 12:17:34.776566 32244 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:34.777734 32217 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:17:34.777889 32217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "2b2eb15d42c24cb98ea5f4ddbcf84e2d"
format_stamp: "Formatted at 2026-08-09 12:17:34 on dist-test-slave-39l8"
server_key: "6055f7c7774da82288d8c27fa624a8a0"
server_key_iv: "8d490ad9aedf88a1da3cce9bc31158d8"
server_key_version: "encryptionkey@0"
I20260809 12:17:34.778072 32217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:34.801728 32217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:34.805518 32217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:34.805809 32217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:34.806694 32217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:34.807981 32217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:34.808053 32217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:34.808204 32217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:34.808269 32217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:34.841454 32217 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:36689
I20260809 12:17:34.841537 32412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:36689 every 8 connection(s)
I20260809 12:17:34.842672 32217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:17:34.848807 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 32217
I20260809 12:17:34.849030 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:17:34.849659 20191 external_mini_cluster.cc:1428] Setting key 4a7fdded5d678208a2f2e8558c0e828a
Aug 09 12:17:34 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277854, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:17:34.862160 32415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:34.845145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:39549 (local address 127.19.183.254:35275)
0809 12:17:34.845554 (+ 409us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:34.845559 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:34.846502 (+ 943us) server_negotiation.cc:407] Connection header received
0809 12:17:34.848302 (+ 1800us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:34.848307 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:34.848410 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:34.848548 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:34.850080 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:34.850957 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:34.852618 (+ 1661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:34.853024 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:34.853197 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:34.856606 (+ 3409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:34.856619 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:34.856623 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:34.856653 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:34.859184 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:34.859843 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:34.859849 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:34.859852 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:34.859973 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:34.860373 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:34.860378 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:34.860381 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:34.860701 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:34.860870 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:34.861539 (+ 669us) server_negotiation.cc:299] Negotiation successful
0809 12:17:34.861778 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"thread_start_us":212,"threads_started":1}
I20260809 12:17:34.863101 32416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:34.845607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.193:39549)
0809 12:17:34.846330 (+ 723us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:34.846374 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:34.848049 (+ 1675us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:34.848731 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:34.848738 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:34.849304 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:34.849900 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:34.849919 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:34.851138 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:34.851142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:34.852468 (+ 1326us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:34.852481 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:34.853301 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:34.853310 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:34.853524 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:34.854144 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:34.854164 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:34.856414 (+ 2250us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:34.859321 (+ 2907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:34.859328 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:34.859340 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:34.859728 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:34.860083 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:34.860088 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:34.860091 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:34.860263 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:34.860840 (+ 577us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:34.860847 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:34.861164 (+ 317us) client_negotiation.cc:769] Sending connection context
0809 12:17:34.861432 (+ 268us) client_negotiation.cc:240] Negotiation successful
0809 12:17:34.861949 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":507,"thread_start_us":291,"threads_started":1}
I20260809 12:17:34.864660 32413 heartbeater.cc:347] Connected to a master server at 127.19.183.254:35275
I20260809 12:17:34.864938 32413 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:34.865631 32413 heartbeater.cc:511] Master 127.19.183.254:35275 requested a full tablet report, sending...
I20260809 12:17:34.867722 32125 ts_manager.cc:194] Registered new tserver with Master: 2b2eb15d42c24cb98ea5f4ddbcf84e2d (127.19.183.193:36689)
I20260809 12:17:34.869138 32125 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:39549
WARNING: no policy specified for kudu/127.19.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
I20260809 12:17:34.909243 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35275
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:33091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:17:35.238094 32420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:35.238370 32420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:35.238468 32420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:35.245862 32420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:35.245968 32420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:35.246093 32420 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:17:35.255792 32420 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:33091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35275
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:35.257153 32420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:35.258523 32420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:35.272670 32425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:35.272708 32430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:35.272751 32426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:35.273571 32420 server_base.cc:1034] running on GCE node
I20260809 12:17:35.274240 32420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:35.275240 32420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:35.276484 32420 hybrid_clock.cc:648] HybridClock initialized: now 1786277855276416 us; error 80 us; skew 500 ppm
I20260809 12:17:35.276824 32420 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:35 dist-test-slave-39l8 krb5kdc[32065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277855, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:35.280155 32420 init.cc:377] Logged in from keytab as kudu/127.19.183.194@KRBTEST.COM (short username kudu)
I20260809 12:17:35.281852 32420 webserver.cc:460] Webserver started at http://127.19.183.194:44787/ using document root <none> and password file <none>
I20260809 12:17:35.282655 32420 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:35.282843 32420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:35.283193 32420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:35.286134 32420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "c793ebbf13a4486fb0852b1512db5ebb"
format_stamp: "Formatted at 2026-08-09 12:17:35 on dist-test-slave-39l8"
server_key: "75390a44745cb3ccd160dc402f78c093"
server_key_iv: "983e2940e1f4eacc27e0dade82dc5275"
server_key_version: "encryptionkey@0"
I20260809 12:17:35.287060 32420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "c793ebbf13a4486fb0852b1512db5ebb"
format_stamp: "Formatted at 2026-08-09 12:17:35 on dist-test-slave-39l8"
server_key: "75390a44745cb3ccd160dc402f78c093"
server_key_iv: "983e2940e1f4eacc27e0dade82dc5275"
server_key_version: "encryptionkey@0"
I20260809 12:17:35.292331 32420 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 12:17:35.295811 32440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:35.297013 32420 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:17:35.297219 32420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "c793ebbf13a4486fb0852b1512db5ebb"
format_stamp: "Formatted at 2026-08-09 12:17:35 on dist-test-slave-39l8"
server_key: "75390a44745cb3ccd160dc402f78c093"
server_key_iv: "983e2940e1f4eacc27e0dade82dc5275"
server_key_version: "encryptionkey@0"
I20260809 12:17:35.297439 32420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:35.314163 32420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:35.318071 32420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:35.318380 32420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:35.319357 32420 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:35.320753 32420 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:35.320873 32420 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:35.321027 32420 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:35.321113 32420 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:35.356030 32420 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:39137
I20260809 12:17:35.356065 32600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:39137 every 8 connection(s)
I20260809 12:17:35.357489 32420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:17:35.363431 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 32420
I20260809 12:17:35.363659 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:17:35.364235 20191 external_mini_cluster.cc:1428] Setting key 5f13206e5e7699e6fb4af66a0552eab9
Aug 09 12:17:35 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277855, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:17:35.381438 32415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:35.360293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:45095 (local address 127.19.183.254:35275)
0809 12:17:35.360481 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:35.360486 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:35.361804 (+ 1318us) server_negotiation.cc:407] Connection header received
0809 12:17:35.363886 (+ 2082us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:35.363893 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:35.363985 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:35.364114 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:35.365736 (+ 1622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:35.366601 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:35.369203 (+ 2602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:35.369646 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:35.369738 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:35.374419 (+ 4681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:35.374428 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:35.374435 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:35.374476 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:35.377906 (+ 3430us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:35.378892 (+ 986us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:35.378900 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:35.378905 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:35.379075 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:35.379501 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:35.379509 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:35.379513 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:35.379887 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:35.380054 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:35.380709 (+ 655us) server_negotiation.cc:299] Negotiation successful
0809 12:17:35.380971 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:17:35.382362 32605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:35.360808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.194:45095)
0809 12:17:35.361599 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:35.361682 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:35.363499 (+ 1817us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:35.364307 (+ 808us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:35.364315 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:35.364863 (+ 548us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:35.365567 (+ 704us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:35.365585 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:35.366827 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:35.366834 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:35.369061 (+ 2227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:35.369079 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:35.369878 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:35.369888 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:35.370231 (+ 343us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:35.371031 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:35.371074 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:35.374191 (+ 3117us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:35.378056 (+ 3865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:35.378067 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:35.378084 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:35.378753 (+ 669us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:35.379201 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:35.379204 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:35.379207 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:35.379379 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:35.380017 (+ 638us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:35.380023 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:35.380320 (+ 297us) client_negotiation.cc:769] Sending connection context
0809 12:17:35.380538 (+ 218us) client_negotiation.cc:240] Negotiation successful
0809 12:17:35.381085 (+ 547us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":360,"threads_started":1}
I20260809 12:17:35.383865 32601 heartbeater.cc:347] Connected to a master server at 127.19.183.254:35275
I20260809 12:17:35.384243 32601 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:35.385167 32601 heartbeater.cc:511] Master 127.19.183.254:35275 requested a full tablet report, sending...
I20260809 12:17:35.386864 32125 ts_manager.cc:194] Registered new tserver with Master: c793ebbf13a4486fb0852b1512db5ebb (127.19.183.194:39137)
I20260809 12:17:35.387673 32125 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:45095
WARNING: no policy specified for kudu/127.19.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
I20260809 12:17:35.430775 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35275
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:33091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:17:35.772902 32610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:35.773257 32610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:35.773442 32610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:35.780753 32610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:35.780859 32610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:35.781087 32610 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:17:35.790875 32610 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:33091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35275
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:35.792219 32610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:35.793567 32610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:35.807471 32623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:35.807569 32622 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:35.807543 32625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:35.808426 32610 server_base.cc:1034] running on GCE node
I20260809 12:17:35.809103 32610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:35.810055 32610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:35.811290 32610 hybrid_clock.cc:648] HybridClock initialized: now 1786277855811240 us; error 39 us; skew 500 ppm
I20260809 12:17:35.811622 32610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:35 dist-test-slave-39l8 krb5kdc[32065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277855, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:35.814839 32610 init.cc:377] Logged in from keytab as kudu/127.19.183.195@KRBTEST.COM (short username kudu)
I20260809 12:17:35.816390 32610 webserver.cc:460] Webserver started at http://127.19.183.195:42941/ using document root <none> and password file <none>
I20260809 12:17:35.817112 32610 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:35.817265 32610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:35.817590 32610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:35.820540 32610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "c08f5e23c5a948cf8645ba6a0a0424ce"
format_stamp: "Formatted at 2026-08-09 12:17:35 on dist-test-slave-39l8"
server_key: "deaa8de62bd5a740e9fc7d57316f090a"
server_key_iv: "2ab296b9ed51cc598ccf645eb7c22c83"
server_key_version: "encryptionkey@0"
I20260809 12:17:35.821354 32610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "c08f5e23c5a948cf8645ba6a0a0424ce"
format_stamp: "Formatted at 2026-08-09 12:17:35 on dist-test-slave-39l8"
server_key: "deaa8de62bd5a740e9fc7d57316f090a"
server_key_iv: "2ab296b9ed51cc598ccf645eb7c22c83"
server_key_version: "encryptionkey@0"
I20260809 12:17:35.826514 32610 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 12:17:35.829981 32639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:35.831358 32610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:17:35.831544 32610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "c08f5e23c5a948cf8645ba6a0a0424ce"
format_stamp: "Formatted at 2026-08-09 12:17:35 on dist-test-slave-39l8"
server_key: "deaa8de62bd5a740e9fc7d57316f090a"
server_key_iv: "2ab296b9ed51cc598ccf645eb7c22c83"
server_key_version: "encryptionkey@0"
I20260809 12:17:35.831769 32610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:35.858006 32610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:35.862270 32610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:35.862572 32610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:35.863606 32610 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:35.864989 32610 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:35.865098 32610 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:35.865264 32610 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:35.865343 32610 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:35.872637 32413 heartbeater.cc:503] Master 127.19.183.254:35275 was elected leader, sending a full tablet report...
I20260809 12:17:35.902909 32610 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:42767
I20260809 12:17:35.902961 330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:42767 every 8 connection(s)
I20260809 12:17:35.904254 32610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:17:35.908782 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 32610
I20260809 12:17:35.908986 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:17:35.909415 20191 external_mini_cluster.cc:1428] Setting key f480a7cc01ff8d6ac3d6577d1b452320
Aug 09 12:17:35 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277855, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:17:35.925669 333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:35.906852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:54419 (local address 127.19.183.254:35275)
0809 12:17:35.907293 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:35.907298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:35.908314 (+ 1016us) server_negotiation.cc:407] Connection header received
0809 12:17:35.910522 (+ 2208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:35.910528 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:35.910626 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:35.910777 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:35.912904 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:35.913716 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:35.915345 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:35.915756 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:35.915861 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:35.919775 (+ 3914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:35.919780 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:35.919783 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:35.919815 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:35.922253 (+ 2438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:35.923012 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:35.923018 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:35.923030 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:35.923209 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:35.923739 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:35.923746 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:35.923749 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:35.924130 (+ 381us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:35.924276 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:35.925160 (+ 884us) server_negotiation.cc:299] Negotiation successful
0809 12:17:35.925392 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":219,"threads_started":1}
I20260809 12:17:35.926669 334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:35.907430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.195:54419)
0809 12:17:35.908149 (+ 719us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:35.908189 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:35.910183 (+ 1994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:35.911055 (+ 872us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:35.911063 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:35.911830 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:35.912698 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:35.912735 (+ 37us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:35.913862 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:35.913866 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:35.915196 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:35.915215 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:35.915964 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:35.915969 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:35.916194 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:35.917031 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:35.917089 (+ 58us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:35.919570 (+ 2481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:35.922379 (+ 2809us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:35.922385 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:35.922398 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:35.922886 (+ 488us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:35.923355 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:17:35.923359 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:17:35.923363 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:17:35.923615 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:17:35.924262 (+ 647us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:35.924268 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:35.924695 (+ 427us) client_negotiation.cc:769] Sending connection context
0809 12:17:35.925022 (+ 327us) client_negotiation.cc:240] Negotiation successful
0809 12:17:35.925522 (+ 500us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":288,"threads_started":1}
I20260809 12:17:35.928061 331 heartbeater.cc:347] Connected to a master server at 127.19.183.254:35275
I20260809 12:17:35.928393 331 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:35.929095 331 heartbeater.cc:511] Master 127.19.183.254:35275 requested a full tablet report, sending...
I20260809 12:17:35.930471 32125 ts_manager.cc:194] Registered new tserver with Master: c08f5e23c5a948cf8645ba6a0a0424ce (127.19.183.195:42767)
I20260809 12:17:35.931300 32125 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:54419
I20260809 12:17:35.937523 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.19.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.193.keytab.
Entry for principal oryx/127.19.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.193.keytab.
WARNING: no policy specified for oryx/127.19.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.194.keytab.
Entry for principal oryx/127.19.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.194.keytab.
WARNING: no policy specified for oryx/127.19.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.19.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.19.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.195.keytab.
Entry for principal oryx/127.19.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.195.keytab.
I20260809 12:17:36.119050 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 32217
Aug 09 12:17:36 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.193@KRBTEST.COM
I20260809 12:17:36.138948 347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:36.120376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46582 (local address 127.19.183.193:36689)
0809 12:17:36.121272 (+ 896us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:36.121279 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:36.121306 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:17:36.121378 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:36.121386 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:36.121602 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:36.121751 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:36.123224 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:36.124258 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:36.125435 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:36.125969 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:36.126061 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:36.133259 (+ 7198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:36.133270 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:36.133284 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:36.133320 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:36.136197 (+ 2877us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:36.136864 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:36.136872 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:36.136874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:36.137004 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:36.137380 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:36.137385 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:36.137388 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:36.137712 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:36.138006 (+ 294us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:36.138317 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 12:17:36.138583 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":754,"thread_start_us":596,"threads_started":1}
I20260809 12:17:36.140241 32406 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:36.224579 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 32217
I20260809 12:17:36.249370 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 32420
Aug 09 12:17:36 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.194@KRBTEST.COM
I20260809 12:17:36.266031 351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:36.250268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52342 (local address 127.19.183.194:39137)
0809 12:17:36.250867 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:36.250875 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:36.250901 (+ 26us) server_negotiation.cc:407] Connection header received
0809 12:17:36.250975 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:36.250981 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:36.251202 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:36.251339 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:36.252631 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:36.253788 (+ 1157us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:36.255125 (+ 1337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:36.255663 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:36.255764 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:36.260133 (+ 4369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:36.260146 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:36.260162 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:36.260198 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:36.263401 (+ 3203us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:36.264085 (+ 684us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:36.264092 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:36.264094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:36.264220 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:36.264578 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:36.264582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:36.264584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:36.264946 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:36.265216 (+ 270us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:36.265497 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 12:17:36.265708 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":445,"thread_start_us":328,"threads_started":1}
I20260809 12:17:36.267469 32591 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:36.350783 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 32420
I20260809 12:17:36.377293 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 32610
Aug 09 12:17:36 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277853, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.195@KRBTEST.COM
I20260809 12:17:36.394605 353 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:36.378213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56288 (local address 127.19.183.195:42767)
0809 12:17:36.378836 (+ 623us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:36.378845 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:36.378877 (+ 32us) server_negotiation.cc:407] Connection header received
0809 12:17:36.378944 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:36.378951 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:36.379179 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:36.379316 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:36.380701 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:36.381685 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:36.383197 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:36.383798 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:36.383911 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:36.388777 (+ 4866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:36.388789 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:36.388805 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:36.388846 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:36.391861 (+ 3015us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:36.392533 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:36.392540 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:36.392542 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:36.392683 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:36.393026 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:36.393029 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:36.393031 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:36.393414 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:36.393652 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:36.393963 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 12:17:36.394218 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":278,"threads_started":1}
I20260809 12:17:36.395984 326 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:36.480999 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 32610
I20260809 12:17:36.509150 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 32082
I20260809 12:17:36.510421 32173 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:36.617370 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 32082
I20260809 12:17:36.649205 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:35275
--webserver_interface=127.19.183.254
--webserver_port=43301
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:33091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:35275
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:17:36.992097 356 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:36.992427 356 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:36.992594 356 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:36.999666 356 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:17:36.999805 356 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:36.999924 356 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:37.000037 356 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:17:37.000119 356 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:17:37.009706 356 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:33091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:35275
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:35275
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=43301
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:37.011008 356 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:37.012400 356 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:37.024075 362 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:37.024020 367 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:37.024106 372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:37.024863 356 server_base.cc:1034] running on GCE node
I20260809 12:17:37.025666 356 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:37.027045 356 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:37.028282 356 hybrid_clock.cc:648] HybridClock initialized: now 1786277857028234 us; error 56 us; skew 500 ppm
I20260809 12:17:37.028631 356 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:37 dist-test-slave-39l8 krb5kdc[32065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277857, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:37.031774 356 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:17:37.033179 356 webserver.cc:460] Webserver started at http://127.19.183.254:43301/ using document root <none> and password file <none>
I20260809 12:17:37.033870 356 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:37.034053 356 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:37.039500 356 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:17:37.042171 382 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:37.043557 356 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:17:37.043762 356 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "48df2a99720c4eb597114faec466d5c9"
format_stamp: "Formatted at 2026-08-09 12:17:34 on dist-test-slave-39l8"
server_key: "f6ffd7f52e712a220505f638fef8425f"
server_key_iv: "3f7c5f93f1e8ebb5ca9d0b4eb7f177e0"
server_key_version: "encryptionkey@0"
I20260809 12:17:37.044775 356 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:37.064096 356 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:37.068517 356 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:37.068794 356 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:37.087692 356 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:35275
I20260809 12:17:37.087728 457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:35275 every 8 connection(s)
I20260809 12:17:37.089237 356 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:17:37.096397 458 sys_catalog.cc:263] Verifying existing consensus state
I20260809 12:17:37.096726 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 356
I20260809 12:17:37.097512 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:36689
--local_ip_for_outbound_sockets=127.19.183.193
--tserver_master_addrs=127.19.183.254:35275
--webserver_port=46315
--webserver_interface=127.19.183.193
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:33091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.183.193
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.193.keytab with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:17:37.099797 458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9: Bootstrap starting.
I20260809 12:17:37.119596 458 log.cc:824] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9: Log is configured to *not* fsync() on all Append() calls
I20260809 12:17:37.128504 458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:17:37.129070 458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9: Bootstrap complete.
I20260809 12:17:37.138247 458 raft_consensus.cc:348] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48df2a99720c4eb597114faec466d5c9" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35275 } }
I20260809 12:17:37.139163 458 raft_consensus.cc:729] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 48df2a99720c4eb597114faec466d5c9, State: Initialized, Role: FOLLOWER
I20260809 12:17:37.139853 458 consensus_queue.cc:260] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48df2a99720c4eb597114faec466d5c9" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35275 } }
I20260809 12:17:37.140110 458 raft_consensus.cc:388] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:17:37.140241 458 raft_consensus.cc:482] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:17:37.140429 458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [term 1 FOLLOWER]: Advancing to term 2
I20260809 12:17:37.144301 458 raft_consensus.cc:504] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48df2a99720c4eb597114faec466d5c9" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35275 } }
I20260809 12:17:37.144809 458 leader_election.cc:302] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 48df2a99720c4eb597114faec466d5c9; no voters:
I20260809 12:17:37.145465 458 leader_election.cc:288] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 12:17:37.145625 463 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 12:17:37.146994 463 raft_consensus.cc:686] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [term 2 LEADER]: Becoming Leader. State: Replica: 48df2a99720c4eb597114faec466d5c9, State: Running, Role: LEADER
I20260809 12:17:37.147657 463 consensus_queue.cc:237] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48df2a99720c4eb597114faec466d5c9" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35275 } }
I20260809 12:17:37.147972 458 sys_catalog.cc:564] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:17:37.150568 468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 48df2a99720c4eb597114faec466d5c9. Latest consensus state: current_term: 2 leader_uuid: "48df2a99720c4eb597114faec466d5c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48df2a99720c4eb597114faec466d5c9" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35275 } } }
I20260809 12:17:37.150979 468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:37.151515 467 sys_catalog.cc:455] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "48df2a99720c4eb597114faec466d5c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "48df2a99720c4eb597114faec466d5c9" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35275 } } }
I20260809 12:17:37.151839 467 sys_catalog.cc:458] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9 [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:37.162310 472 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:17:37.169220 472 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:17:37.170830 472 catalog_manager.cc:1259] Loaded cluster ID: 534542138b964705aca00b43d5b1a1b9
I20260809 12:17:37.171015 472 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:17:37.176116 472 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:17:37.178057 472 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 48df2a99720c4eb597114faec466d5c9: Loaded TSK: 0
I20260809 12:17:37.179915 472 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 12:17:37.478312 461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:37.478585 461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:37.478757 461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:37.485939 461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:37.486054 461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:37.486227 461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:17:37.495934 461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:33091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.193.keytab
--principal=oryx/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:36689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=46315
--tserver_master_addrs=127.19.183.254:35275
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:37.497216 461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:37.498458 461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:37.516671 499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:37.516644 502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:37.517100 505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:37.517457 461 server_base.cc:1034] running on GCE node
I20260809 12:17:37.518356 461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:37.519577 461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:37.520830 461 hybrid_clock.cc:648] HybridClock initialized: now 1786277857520760 us; error 55 us; skew 500 ppm
I20260809 12:17:37.521255 461 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:37 dist-test-slave-39l8 krb5kdc[32065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277857, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:37.525349 461 init.cc:377] Logged in from keytab as oryx/127.19.183.193@KRBTEST.COM (short username oryx)
I20260809 12:17:37.527221 461 webserver.cc:460] Webserver started at http://127.19.183.193:46315/ using document root <none> and password file <none>
I20260809 12:17:37.528185 461 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:37.528348 461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:37.535637 461 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.001s
I20260809 12:17:37.538918 514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:37.540428 461 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:17:37.540611 461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "2b2eb15d42c24cb98ea5f4ddbcf84e2d"
format_stamp: "Formatted at 2026-08-09 12:17:34 on dist-test-slave-39l8"
server_key: "6055f7c7774da82288d8c27fa624a8a0"
server_key_iv: "8d490ad9aedf88a1da3cce9bc31158d8"
server_key_version: "encryptionkey@0"
I20260809 12:17:37.541915 461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:37.571949 461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:37.576648 461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:37.576927 461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:37.578028 461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:37.579591 461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:37.579711 461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:37.579845 461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:37.579934 461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:37.621263 461 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:36689
I20260809 12:17:37.621327 679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:36689 every 8 connection(s)
I20260809 12:17:37.622885 461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:17:37.627835 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 461
I20260809 12:17:37.628461 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:39137
--local_ip_for_outbound_sockets=127.19.183.194
--tserver_master_addrs=127.19.183.254:35275
--webserver_port=44787
--webserver_interface=127.19.183.194
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:33091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.183.194
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.194.keytab with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:17:37 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:37 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:37.649735 683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:37.626085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.193:60177)
0809 12:17:37.626953 (+ 868us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:37.627000 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:37.633315 (+ 6315us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:37.635274 (+ 1959us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:37.635288 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:37.636144 (+ 856us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:37.637007 (+ 863us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:37.637037 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:37.639801 (+ 2764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:37.639808 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:37.641961 (+ 2153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:37.641975 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:37.643128 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:37.643136 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:37.643415 (+ 279us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:37.644120 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:37.644144 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:37.648516 (+ 4372us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":367,"threads_started":1}
W20260809 12:17:37.652382 680 heartbeater.cc:644] Failed to heartbeat to 127.19.183.254:35275 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.183.254:35275: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
W20260809 12:17:37.654167 685 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:37.626192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:60177 (local address 127.19.183.254:35275)
0809 12:17:37.631434 (+ 5242us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:37.631446 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:37.631533 (+ 87us) server_negotiation.cc:407] Connection header received
0809 12:17:37.634045 (+ 2512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:37.634068 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:37.634587 (+ 519us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:37.635060 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:37.637260 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:37.639582 (+ 2322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:37.642114 (+ 2532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:37.642856 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:37.643079 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:37.652791 (+ 9712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.193:60177: BlockingRecv error: recv got EOF from 127.19.183.193:60177 (error 108)
Metrics: {"server-negotiator.queue_time_us":4986,"thread_start_us":4764,"threads_started":1}
Aug 09 12:17:37 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:37 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 12:17:37.668931 685 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:37.653551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:50409 (local address 127.19.183.254:35275)
0809 12:17:37.654558 (+ 1007us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:37.654563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:37.655094 (+ 531us) server_negotiation.cc:407] Connection header received
0809 12:17:37.655381 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:37.655389 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:37.655487 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:37.655587 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:37.656931 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:37.658420 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:37.660837 (+ 2417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:37.661743 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:37.662194 (+ 451us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:37.668553 (+ 6359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.193:50409: BlockingRecv error: recv got EOF from 127.19.183.193:50409 (error 108)
Metrics: {"server-negotiator.queue_time_us":881,"threads_started":1}
I20260809 12:17:37.669565 683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:37.653176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.193:50409)
0809 12:17:37.654987 (+ 1811us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:37.655009 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:37.655216 (+ 207us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:37.655842 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:37.655849 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:37.656432 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:37.656686 (+ 254us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:37.656701 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:37.658679 (+ 1978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:37.658687 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:37.660627 (+ 1940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:37.660646 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:37.662565 (+ 1919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:37.662574 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:37.662841 (+ 267us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:37.663679 (+ 838us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:37.663711 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:37.669144 (+ 5433us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":1668,"mutex_wait_us":1485}
Aug 09 12:17:37 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:37 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:37.678841 683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:37.670362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.193:59885)
0809 12:17:37.670660 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:37.670680 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:37.670804 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:37.671143 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:37.671150 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:37.671619 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:37.671783 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:37.671792 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:37.672925 (+ 1133us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:37.672931 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:37.673854 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:37.673865 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:37.674610 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:37.674616 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:37.674829 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:37.675447 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:37.675464 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:37.678488 (+ 3024us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":191}
W20260809 12:17:37.679363 680 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 12:17:37.679111 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:37.670637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:59885 (local address 127.19.183.254:35275)
0809 12:17:37.670819 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:37.670825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:37.670862 (+ 37us) server_negotiation.cc:407] Connection header received
0809 12:17:37.670913 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:37.670918 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:37.670985 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:37.671119 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:37.671928 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:37.672778 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:37.674000 (+ 1222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:37.674458 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:37.674596 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:37.678772 (+ 4176us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.193:59885: BlockingRecv error: recv got EOF from 127.19.183.193:59885 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 12:17:38.020742 688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:38.021003 688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:38.021096 688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:38.028185 688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:38.028303 688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:38.028414 688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:17:38.037892 688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:33091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.194.keytab
--principal=oryx/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:39137
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=44787
--tserver_master_addrs=127.19.183.254:35275
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:38.039311 688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:38.040573 688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:38.053926 697 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:38.054116 696 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:38.053958 699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:38.055178 688 server_base.cc:1034] running on GCE node
I20260809 12:17:38.055800 688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:38.056726 688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:38.057930 688 hybrid_clock.cc:648] HybridClock initialized: now 1786277858057881 us; error 38 us; skew 500 ppm
I20260809 12:17:38.058303 688 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:38 dist-test-slave-39l8 krb5kdc[32065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277858, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:38.061460 688 init.cc:377] Logged in from keytab as oryx/127.19.183.194@KRBTEST.COM (short username oryx)
I20260809 12:17:38.063117 688 webserver.cc:460] Webserver started at http://127.19.183.194:44787/ using document root <none> and password file <none>
I20260809 12:17:38.063788 688 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:38.063946 688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:38.069407 688 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.003s
I20260809 12:17:38.072301 708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:38.073707 688 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:17:38.073915 688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "c793ebbf13a4486fb0852b1512db5ebb"
format_stamp: "Formatted at 2026-08-09 12:17:35 on dist-test-slave-39l8"
server_key: "75390a44745cb3ccd160dc402f78c093"
server_key_iv: "983e2940e1f4eacc27e0dade82dc5275"
server_key_version: "encryptionkey@0"
I20260809 12:17:38.074966 688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:38.093825 688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:38.098747 688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:38.099004 688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:38.100116 688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:38.101701 688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:38.101801 688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:38.101883 688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:38.101961 688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:38.136001 688 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:39137
I20260809 12:17:38.136085 877 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:39137 every 8 connection(s)
I20260809 12:17:38.137405 688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:17:38.146168 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 688
I20260809 12:17:38.146862 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:42767
--local_ip_for_outbound_sockets=127.19.183.195
--tserver_master_addrs=127.19.183.254:35275
--webserver_port=42941
--webserver_interface=127.19.183.195
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:33091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.19.183.195
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.195.keytab with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:17:38 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:38 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 12:17:38.154160 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:38.139968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:41209 (local address 127.19.183.254:35275)
0809 12:17:38.140166 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:38.140171 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:38.141232 (+ 1061us) server_negotiation.cc:407] Connection header received
0809 12:17:38.143166 (+ 1934us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:38.143171 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:38.143260 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:38.143369 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:38.144794 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.145499 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.147482 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.147975 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.148110 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:38.153985 (+ 5875us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.194:41209: BlockingRecv error: recv got EOF from 127.19.183.194:41209 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 12:17:38.155092 882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:38.140359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.194:41209)
0809 12:17:38.141046 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:38.141081 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:38.142879 (+ 1798us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:38.143604 (+ 725us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:38.143611 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:38.144112 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:38.144632 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:38.144649 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.145730 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.145734 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:38.147334 (+ 1600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:38.147347 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.148209 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.148217 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:38.148565 (+ 348us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:38.149301 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:38.149335 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:38.153832 (+ 4497us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":268,"threads_started":1}
W20260809 12:17:38.155755 878 heartbeater.cc:644] Failed to heartbeat to 127.19.183.254:35275 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.183.254:35275: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Aug 09 12:17:38 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:38 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:38.165202 882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:38.156328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.194:49123)
0809 12:17:38.156529 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:38.156549 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:38.156648 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:38.156971 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:38.156977 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:38.157425 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:38.157627 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:38.157642 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.158758 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.158765 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:38.159683 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:38.159695 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.160703 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.160708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:38.160822 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:38.161433 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:38.161457 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:38.164867 (+ 3410us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 12:17:38.165217 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:38.156462 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:49123 (local address 127.19.183.254:35275)
0809 12:17:38.156596 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:38.156601 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:38.156616 (+ 15us) server_negotiation.cc:407] Connection header received
0809 12:17:38.156772 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:38.156776 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:38.156829 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:38.156914 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:38.157788 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.158590 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.159832 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.160561 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.160704 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:38.165038 (+ 4334us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.194:49123: BlockingRecv error: recv got EOF from 127.19.183.194:49123 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
Aug 09 12:17:38 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:38 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 12:17:38.174163 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:38.165968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:60301 (local address 127.19.183.254:35275)
0809 12:17:38.166122 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:38.166127 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:38.166145 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:17:38.166291 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:38.166296 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:38.166363 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:38.166463 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:38.167151 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.167847 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.169026 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.169476 (+ 450us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.169607 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:38.173992 (+ 4385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.194:60301: BlockingRecv error: recv got EOF from 127.19.183.194:60301 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 12:17:38.174217 882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:38.165876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.194:60301)
0809 12:17:38.166031 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:38.166050 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:38.166161 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:38.166490 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:38.166494 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:38.166872 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:38.167011 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:38.167019 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.167990 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.167996 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:38.168883 (+ 887us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:38.168894 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.169620 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.169626 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:38.169824 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:38.170449 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:38.170465 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:38.173849 (+ 3384us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":40}
W20260809 12:17:38.174480 878 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 12:17:38.488377 888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:38.488659 888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:38.488756 888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:38.495736 888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:38.495843 888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:38.495960 888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:17:38.505460 888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:33091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/krb5kdc/oryx_127.19.183.195.keytab
--principal=oryx/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:42767
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=42941
--tserver_master_addrs=127.19.183.254:35275
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:38.506870 888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:38.508219 888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:38.521450 894 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:38.521476 895 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:38.521427 897 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:38.522583 888 server_base.cc:1034] running on GCE node
I20260809 12:17:38.523252 888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:38.524294 888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:38.525568 888 hybrid_clock.cc:648] HybridClock initialized: now 1786277858525511 us; error 61 us; skew 500 ppm
I20260809 12:17:38.525915 888 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:17:38 dist-test-slave-39l8 krb5kdc[32065](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277858, etypes {rep=17 tkt=17 ses=17}, oryx/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:17:38.529567 888 init.cc:377] Logged in from keytab as oryx/127.19.183.195@KRBTEST.COM (short username oryx)
I20260809 12:17:38.531046 888 webserver.cc:460] Webserver started at http://127.19.183.195:42941/ using document root <none> and password file <none>
I20260809 12:17:38.531777 888 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:38.531945 888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:38.537390 888 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.001s
I20260809 12:17:38.540110 904 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:38.541401 888 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 12:17:38.541572 888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "c08f5e23c5a948cf8645ba6a0a0424ce"
format_stamp: "Formatted at 2026-08-09 12:17:35 on dist-test-slave-39l8"
server_key: "deaa8de62bd5a740e9fc7d57316f090a"
server_key_iv: "2ab296b9ed51cc598ccf645eb7c22c83"
server_key_version: "encryptionkey@0"
I20260809 12:17:38.542544 888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:38.561381 888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:38.565128 888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:38.565402 888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:38.567412 888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:38.568722 888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:38.568838 888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:38.568977 888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:38.569077 888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:38.603216 888 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:42767
I20260809 12:17:38.603341 1073 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:42767 every 8 connection(s)
I20260809 12:17:38.604743 888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:17:38.608726 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 888
Aug 09 12:17:38 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:38 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
W20260809 12:17:38.622483 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:38.607296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:56241 (local address 127.19.183.254:35275)
0809 12:17:38.607480 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:38.607485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:38.608650 (+ 1165us) server_negotiation.cc:407] Connection header received
0809 12:17:38.611621 (+ 2971us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:38.611629 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:38.611721 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:38.611835 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:38.613389 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.614276 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.616180 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.616840 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.616972 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:38.622170 (+ 5198us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.195:56241: BlockingRecv error: recv got EOF from 127.19.183.195:56241 (error 108)
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":67}
I20260809 12:17:38.623329 1078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:38.607709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.195:56241)
0809 12:17:38.608440 (+ 731us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:38.608507 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:38.611299 (+ 2792us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:38.612062 (+ 763us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:38.612070 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:38.612550 (+ 480us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:38.613211 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:38.613235 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.614446 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.614450 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:38.615985 (+ 1535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:38.616002 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.617060 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.617065 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:38.617264 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:38.617905 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:38.617926 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:38.621950 (+ 4024us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":252,"threads_started":1}
W20260809 12:17:38.623962 1076 heartbeater.cc:644] Failed to heartbeat to 127.19.183.254:35275 (0 consecutive failures): Not authorized: Failed to ping master at 127.19.183.254:35275: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
I20260809 12:17:38.627764 1080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:38.609961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51662 (local address 127.19.183.254:35275)
0809 12:17:38.611065 (+ 1104us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:38.611071 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:38.611092 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:17:38.611156 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:38.611161 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:38.611236 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:38.611367 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:38.613022 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.613901 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.615426 (+ 1525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.616058 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.616183 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:38.619033 (+ 2850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:38.619045 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:38.619070 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:38.619123 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:38.622950 (+ 3827us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:38.624245 (+ 1295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:38.624252 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:38.624279 (+ 27us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:38.624496 (+ 217us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:17:38.625017 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:17:38.625022 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:17:38.625026 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:17:38.625565 (+ 539us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:38.625865 (+ 300us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:38.627248 (+ 1383us) server_negotiation.cc:299] Negotiation successful
0809 12:17:38.627496 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":984,"thread_start_us":831,"threads_started":1}
Aug 09 12:17:38 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:38 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:38.632632 1078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:38.624561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.195:54267)
0809 12:17:38.624733 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:38.624752 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:38.624845 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:38.625251 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:38.625255 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:38.625648 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:38.625801 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:38.625811 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.627112 (+ 1301us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.627120 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:38.628128 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:38.628141 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.628831 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.628837 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:38.629020 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:38.629538 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:38.629553 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:38.632356 (+ 2803us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 12:17:38.632747 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:38.624757 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:54267 (local address 127.19.183.254:35275)
0809 12:17:38.624958 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:38.624964 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:38.624982 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:17:38.625022 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:38.625026 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:38.625109 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:38.625211 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:38.625994 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.626969 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.628267 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.628692 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.628823 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:38.632539 (+ 3716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.195:54267: BlockingRecv error: recv got EOF from 127.19.183.195:54267 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 12:17:38 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:38 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:38.640401 1078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:38.633253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.195:56401)
0809 12:17:38.633418 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:38.633451 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:38.633549 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:38.633832 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:38.633836 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:38.634123 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:38.634268 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:38.634275 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.635295 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.635300 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:38.636148 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:38.636158 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.636851 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.636858 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:38.637035 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:38.637483 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:38.637535 (+ 52us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:38.640103 (+ 2568us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 12:17:38.640571 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:38.633357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:56401 (local address 127.19.183.254:35275)
0809 12:17:38.633575 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:38.633583 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:38.633599 (+ 16us) server_negotiation.cc:407] Connection header received
0809 12:17:38.633647 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:38.633652 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:38.633711 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:38.633806 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:38.634413 (+ 607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.635154 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.636302 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.636704 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.636828 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:38.640311 (+ 3483us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.195:56401: BlockingRecv error: recv got EOF from 127.19.183.195:56401 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
W20260809 12:17:38.640795 1076 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 12:17:38 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:38 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:38.687871 1081 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:38.680006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.193:34683)
0809 12:17:38.680489 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:38.680505 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:38.680628 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:38.680927 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:38.680930 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:38.681394 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:38.681589 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:38.681599 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.682550 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.682554 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:38.683521 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:38.683531 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.684218 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.684223 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:38.684339 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:38.684915 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:38.684932 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:38.687644 (+ 2712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":271,"threads_started":1}
W20260809 12:17:38.688035 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:38.680054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:34683 (local address 127.19.183.254:35275)
0809 12:17:38.680307 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:38.680314 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:38.680596 (+ 282us) server_negotiation.cc:407] Connection header received
0809 12:17:38.680737 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:38.680741 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:38.680804 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:38.680885 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:38.681719 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.682432 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.683650 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:38.684066 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:38.684157 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:38.687837 (+ 3680us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.193:34683: BlockingRecv error: recv got EOF from 127.19.183.193:34683 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 09 12:17:39 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:39 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:39.184459 1082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:39.175310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.194:38667)
0809 12:17:39.175969 (+ 659us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:39.175993 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:39.176167 (+ 174us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:39.176524 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:39.176529 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:39.177027 (+ 498us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:39.177235 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:39.177245 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:39.178367 (+ 1122us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:39.178372 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:39.179553 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:39.179566 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:39.180338 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:39.180346 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:39.180509 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:39.181166 (+ 657us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:39.181184 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:39.184097 (+ 2913us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":311,"threads_started":1}
W20260809 12:17:39.184540 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:39.175385 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:38667 (local address 127.19.183.254:35275)
0809 12:17:39.175661 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:39.175671 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:39.176116 (+ 445us) server_negotiation.cc:407] Connection header received
0809 12:17:39.176308 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:39.176312 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:39.176378 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:39.176491 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:39.177382 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:39.178218 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:39.179714 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:39.180142 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:39.180274 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:39.184294 (+ 4020us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.194:38667: BlockingRecv error: recv got EOF from 127.19.183.194:38667 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
Aug 09 12:17:39 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:39 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:39.650779 1085 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:39.641495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.195:60055)
0809 12:17:39.642067 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:39.642083 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:39.642217 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:39.642524 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:39.642527 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:39.643154 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:39.643449 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:39.643469 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:39.644532 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:39.644538 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:39.645620 (+ 1082us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:39.645635 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:39.646521 (+ 886us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:39.646527 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:39.646743 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:39.647343 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:39.647361 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:39.650502 (+ 3141us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":322,"threads_started":1}
W20260809 12:17:39.651000 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:39.641595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:60055 (local address 127.19.183.254:35275)
0809 12:17:39.641801 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:39.641805 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:39.642176 (+ 371us) server_negotiation.cc:407] Connection header received
0809 12:17:39.642322 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:39.642325 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:39.642389 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:39.642476 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:39.643633 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:39.644382 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:39.645785 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:39.646309 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:39.646450 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:39.650753 (+ 4303us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.195:60055: BlockingRecv error: recv got EOF from 127.19.183.195:60055 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 09 12:17:39 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:39 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:39.697587 1086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:39.688933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.193:47075)
0809 12:17:39.689482 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:39.689497 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:39.689675 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:39.690028 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:39.690033 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:39.690628 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:39.690886 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:39.690896 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:39.692247 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:39.692253 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:39.693161 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:39.693173 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:39.693863 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:39.693867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:39.693975 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:39.694560 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:39.694577 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:39.697339 (+ 2762us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":297,"threads_started":1}
W20260809 12:17:39.697767 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:39.689025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:47075 (local address 127.19.183.254:35275)
0809 12:17:39.689262 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:39.689270 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:39.689589 (+ 319us) server_negotiation.cc:407] Connection header received
0809 12:17:39.689815 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:39.689826 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:39.689891 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:39.689993 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:39.691047 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:39.692071 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:39.693302 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:39.693718 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:39.693810 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:39.697514 (+ 3704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.193:47075: BlockingRecv error: recv got EOF from 127.19.183.193:47075 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 09 12:17:40 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:40 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:40.193313 1088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:40.185463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.194:36553)
0809 12:17:40.186004 (+ 541us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:40.186021 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:40.186177 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:40.186448 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:40.186451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:40.186874 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:40.187053 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:40.187062 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:40.188053 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:40.188057 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:40.188957 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:40.188969 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:40.189621 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:40.189625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:40.189738 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:40.190311 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:40.190329 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:40.193019 (+ 2690us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":439,"thread_start_us":322,"threads_started":1}
W20260809 12:17:40.193467 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:40.185566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:36553 (local address 127.19.183.254:35275)
0809 12:17:40.185825 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:40.185832 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:40.186112 (+ 280us) server_negotiation.cc:407] Connection header received
0809 12:17:40.186289 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:40.186293 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:40.186354 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:40.186433 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:40.187202 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:40.187936 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:40.189082 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:40.189490 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:40.189581 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:40.193234 (+ 3653us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.194:36553: BlockingRecv error: recv got EOF from 127.19.183.194:36553 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 09 12:17:40 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:40 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:40.660818 1089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:40.651719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.195:41015)
0809 12:17:40.652334 (+ 615us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:40.652351 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:40.652517 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:40.652884 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:40.652889 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:40.653448 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:40.653622 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:40.653633 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:40.654682 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:40.654687 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:40.655669 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:40.655681 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:40.656693 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:40.656700 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:40.656876 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:40.657523 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:40.657542 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:40.660450 (+ 2908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":522,"thread_start_us":350,"threads_started":1}
W20260809 12:17:40.660928 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:40.651903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:41015 (local address 127.19.183.254:35275)
0809 12:17:40.652174 (+ 271us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:40.652181 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:40.652440 (+ 259us) server_negotiation.cc:407] Connection header received
0809 12:17:40.652655 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:40.652659 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:40.652723 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:40.652829 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:40.653769 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:40.654516 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:40.655818 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:40.656499 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:40.656675 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:40.660668 (+ 3993us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.195:41015: BlockingRecv error: recv got EOF from 127.19.183.195:41015 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 09 12:17:40 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:40 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:40.707365 1090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:40.698601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.193:38669)
0809 12:17:40.699212 (+ 611us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:40.699235 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:40.699389 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:40.699803 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:40.699810 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:40.700330 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:40.700500 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:40.700510 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:40.701501 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:40.701504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:40.702394 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:40.702405 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:40.703327 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:40.703335 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:40.703574 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:40.704262 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:40.704279 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:40.707105 (+ 2826us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":429,"thread_start_us":297,"threads_started":1}
W20260809 12:17:40.707530 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:40.698679 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:38669 (local address 127.19.183.254:35275)
0809 12:17:40.698919 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:40.698928 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:40.699330 (+ 402us) server_negotiation.cc:407] Connection header received
0809 12:17:40.699537 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:40.699544 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:40.699649 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:40.699728 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:40.700633 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:40.701376 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:40.702515 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:40.703176 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:40.703326 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:40.707291 (+ 3965us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.193:38669: BlockingRecv error: recv got EOF from 127.19.183.193:38669 (error 108)
Metrics: {"server-negotiator.queue_time_us":83}
Aug 09 12:17:41 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:41 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:41.210476 1091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:41.194469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.194:59865)
0809 12:17:41.195139 (+ 670us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:41.195173 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:41.195691 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:41.196118 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:41.196123 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:41.197146 (+ 1023us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:41.197618 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:41.197650 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:41.199062 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:41.199068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:41.200167 (+ 1099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:41.200188 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:41.201641 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:41.201650 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:41.201843 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:41.206273 (+ 4430us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:41.206301 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:41.210088 (+ 3787us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":326,"threads_started":1}
W20260809 12:17:41.210533 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:41.194598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:59865 (local address 127.19.183.254:35275)
0809 12:17:41.194860 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:41.194869 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:41.195286 (+ 417us) server_negotiation.cc:407] Connection header received
0809 12:17:41.195854 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:41.195860 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:41.195959 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:41.196067 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:41.197809 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:41.198903 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:41.200387 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:41.201433 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:41.201604 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:41.210315 (+ 8711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.194:59865: BlockingRecv error: recv got EOF from 127.19.183.194:59865 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 09 12:17:41 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:41 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:41.669994 1092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:41.661807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.195:39031)
0809 12:17:41.662350 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:41.662366 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:41.662481 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:41.662786 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:41.662789 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:41.663141 (+ 352us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:41.663314 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:41.663324 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:41.664296 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:41.664299 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:41.665387 (+ 1088us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:41.665401 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:41.666066 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:41.666070 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:41.666205 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:41.666875 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:41.666892 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:41.669740 (+ 2848us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":291,"threads_started":1}
W20260809 12:17:41.670219 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:41.661910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:39031 (local address 127.19.183.254:35275)
0809 12:17:41.662078 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:41.662083 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:41.662439 (+ 356us) server_negotiation.cc:407] Connection header received
0809 12:17:41.662577 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:41.662581 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:41.662651 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:41.662751 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:41.663442 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:41.664181 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:41.665529 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:41.665937 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:41.666038 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:41.669950 (+ 3912us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.195:39031: BlockingRecv error: recv got EOF from 127.19.183.195:39031 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Aug 09 12:17:41 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:41 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:41.716624 1093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:41.708477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.193:46451)
0809 12:17:41.708941 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:41.708957 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:41.709076 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:41.709391 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:41.709394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:41.709744 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:41.709908 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:41.709917 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:41.711047 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:41.711055 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:41.712086 (+ 1031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:41.712098 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:41.712719 (+ 621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:41.712723 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:41.712837 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:41.713468 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:41.713486 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:41.716343 (+ 2857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":257,"threads_started":1}
W20260809 12:17:41.716879 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:41.708522 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:46451 (local address 127.19.183.254:35275)
0809 12:17:41.708780 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:41.708786 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:41.709041 (+ 255us) server_negotiation.cc:407] Connection header received
0809 12:17:41.709180 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:41.709184 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:41.709247 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:41.709326 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:41.710042 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:41.710921 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:41.712223 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:41.712598 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:41.712696 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:41.716599 (+ 3903us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.193:46451: BlockingRecv error: recv got EOF from 127.19.183.193:46451 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 12:17:42 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:42 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:42.219918 1096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:42.211509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.194:36319)
0809 12:17:42.212020 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:42.212037 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:42.212167 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:42.212474 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:42.212477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:42.212853 (+ 376us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:42.213007 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:42.213016 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:42.213975 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:42.213979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:42.215022 (+ 1043us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:42.215040 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:42.215679 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:42.215683 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:42.215801 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:42.216454 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:42.216473 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:42.219570 (+ 3097us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":275,"threads_started":1}
W20260809 12:17:42.219993 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:42.211573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:36319 (local address 127.19.183.254:35275)
0809 12:17:42.211815 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:42.211823 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:42.212126 (+ 303us) server_negotiation.cc:407] Connection header received
0809 12:17:42.212282 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:42.212285 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:42.212348 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:42.212440 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:42.213146 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:42.213854 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:42.215163 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:42.215554 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:42.215644 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:42.219765 (+ 4121us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.194:36319: BlockingRecv error: recv got EOF from 127.19.183.194:36319 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 09 12:17:42 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:42 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:42.679239 1098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:42.671025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.195:34259)
0809 12:17:42.671573 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:42.671589 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:42.671709 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:42.672073 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:42.672077 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:42.672446 (+ 369us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:42.672595 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:42.672606 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:42.673543 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:42.673547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:42.674529 (+ 982us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:42.674541 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:42.675376 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:42.675380 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:42.675502 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:42.676051 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:42.676068 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:42.679013 (+ 2945us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":421,"thread_start_us":305,"threads_started":1}
W20260809 12:17:42.679531 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:42.671049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:34259 (local address 127.19.183.254:35275)
0809 12:17:42.671268 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:42.671273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:42.671672 (+ 399us) server_negotiation.cc:407] Connection header received
0809 12:17:42.671821 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:42.671825 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:42.671888 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:42.671969 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:42.672733 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:42.673432 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:42.674659 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:42.675248 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:42.675340 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:42.679209 (+ 3869us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.195:34259: BlockingRecv error: recv got EOF from 127.19.183.195:34259 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 12:17:42 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:42 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:42.726043 1099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:42.717651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.193:48049)
0809 12:17:42.718264 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:42.718291 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:42.718455 (+ 164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:42.718837 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:42.718842 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:42.719285 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:42.719453 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:42.719462 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:42.720487 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:42.720492 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:42.721457 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:42.721470 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:42.722186 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:42.722190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:42.722323 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:42.722934 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:42.722952 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:42.725780 (+ 2828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":269,"threads_started":1}
W20260809 12:17:42.726133 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:42.717721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:48049 (local address 127.19.183.254:35275)
0809 12:17:42.717931 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:42.717939 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:42.718397 (+ 458us) server_negotiation.cc:407] Connection header received
0809 12:17:42.718595 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:42.718600 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:42.718676 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:42.718794 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:42.719606 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:42.720337 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:42.721590 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:42.722034 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:42.722148 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:42.725962 (+ 3814us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.193:48049: BlockingRecv error: recv got EOF from 127.19.183.193:48049 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 12:17:43 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:43 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:43.229895 1100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:43.220966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.194:59499)
0809 12:17:43.221604 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:43.221631 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:43.221794 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:43.222156 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:43.222161 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:43.222653 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:43.222891 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:43.222901 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:43.223974 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:43.223980 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:43.224914 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:43.224926 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:43.225639 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:43.225645 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:43.225832 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:43.226507 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:43.226525 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:43.229522 (+ 2997us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":519,"thread_start_us":354,"threads_started":1}
W20260809 12:17:43.230010 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:43.221264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:59499 (local address 127.19.183.254:35275)
0809 12:17:43.221504 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:43.221511 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:43.221724 (+ 213us) server_negotiation.cc:407] Connection header received
0809 12:17:43.221935 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:43.221942 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:43.222013 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:43.222117 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:43.223038 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:43.223826 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:43.225063 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:43.225479 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:43.225615 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:43.229742 (+ 4127us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.194:59499: BlockingRecv error: recv got EOF from 127.19.183.194:59499 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
Aug 09 12:17:43 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:43 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:43.688886 1102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:43.680194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.195:48365)
0809 12:17:43.680692 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:43.680709 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:43.680855 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:43.681237 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:43.681241 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:43.681853 (+ 612us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:43.682106 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:43.682118 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:43.683377 (+ 1259us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:43.683383 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:43.684291 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:43.684302 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:43.685058 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:43.685064 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:43.685265 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:43.685862 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:43.685880 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:43.688610 (+ 2730us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":241,"threads_started":1}
W20260809 12:17:43.689033 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:43.680336 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:48365 (local address 127.19.183.254:35275)
0809 12:17:43.680618 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:43.680626 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:43.680837 (+ 211us) server_negotiation.cc:407] Connection header received
0809 12:17:43.680980 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:43.680985 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:43.681083 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:43.681201 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:43.682275 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:43.683225 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:43.684445 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:43.684892 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:43.685032 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:43.688823 (+ 3791us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.195:48365: BlockingRecv error: recv got EOF from 127.19.183.195:48365 (error 108)
Metrics: {"server-negotiator.queue_time_us":102}
Aug 09 12:17:43 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:43 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:43.735483 1103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:43.727027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.193:34289)
0809 12:17:43.727605 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:43.727629 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:43.727745 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:43.728020 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:43.728023 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:43.728356 (+ 333us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:43.728526 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:43.728534 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:43.729678 (+ 1144us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:43.729683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:43.730972 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:43.730986 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:43.731714 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:43.731720 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:43.731893 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:43.732547 (+ 654us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:43.732564 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:43.735264 (+ 2700us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":327,"threads_started":1}
W20260809 12:17:43.735819 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:43.727139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:34289 (local address 127.19.183.254:35275)
0809 12:17:43.727318 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:43.727323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:43.727699 (+ 376us) server_negotiation.cc:407] Connection header received
0809 12:17:43.727840 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:43.727843 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:43.727901 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:43.727980 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:43.728652 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:43.729525 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:43.731118 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:43.731550 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:43.731687 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:43.735452 (+ 3765us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.193:34289: BlockingRecv error: recv got EOF from 127.19.183.193:34289 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 09 12:17:44 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:44 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:44.239861 1104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:44.231033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.194:58401)
0809 12:17:44.231622 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:44.231639 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:44.231807 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:44.232200 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:44.232206 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:44.232628 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:44.232798 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:44.232808 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:44.233873 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:44.233884 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:44.234825 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:44.234837 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:44.235651 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:44.235658 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:44.235837 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:44.236518 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:44.236539 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:44.239515 (+ 2976us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":308,"threads_started":1}
W20260809 12:17:44.240036 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:44.231137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:58401 (local address 127.19.183.254:35275)
0809 12:17:44.231373 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:44.231381 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:44.231734 (+ 353us) server_negotiation.cc:407] Connection header received
0809 12:17:44.231950 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:44.231959 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:44.232051 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:44.232161 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:44.232955 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:44.233717 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:44.234981 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:44.235461 (+ 480us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:44.235609 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:44.239773 (+ 4164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.194:58401: BlockingRecv error: recv got EOF from 127.19.183.194:58401 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 09 12:17:44 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:44 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:44.701581 1105 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:44.689969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.195:36921)
0809 12:17:44.690586 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:44.690612 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:44.690802 (+ 190us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:44.691455 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:44.691463 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:44.692132 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:44.692414 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:44.692432 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:44.693572 (+ 1140us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:44.693587 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:44.695585 (+ 1998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:44.695607 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:44.696443 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:44.696453 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:44.696729 (+ 276us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:44.697479 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:44.697511 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:44.701223 (+ 3712us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":312,"threads_started":1}
W20260809 12:17:44.701793 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:44.690050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:36921 (local address 127.19.183.254:35275)
0809 12:17:44.690339 (+ 289us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:44.690348 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:44.690736 (+ 388us) server_negotiation.cc:407] Connection header received
0809 12:17:44.690990 (+ 254us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:44.690998 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:44.691104 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:44.691190 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:44.692641 (+ 1451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:44.693418 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:44.695781 (+ 2363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:44.696250 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:44.696360 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:44.701484 (+ 5124us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.195:36921: BlockingRecv error: recv got EOF from 127.19.183.195:36921 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 09 12:17:44 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:44 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:44.745615 1106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:44.736462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.193:42829)
0809 12:17:44.737145 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:44.737161 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:44.737339 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:44.737753 (+ 414us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:44.737759 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:44.738246 (+ 487us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:44.738426 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:44.738436 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:44.739650 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:44.739655 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:44.740881 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:44.740914 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:44.741812 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:44.741817 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:44.741937 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:44.742519 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:44.742536 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:44.745351 (+ 2815us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":270,"threads_started":1}
W20260809 12:17:44.746065 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:44.736651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:42829 (local address 127.19.183.254:35275)
0809 12:17:44.736915 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:44.736924 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:44.737260 (+ 336us) server_negotiation.cc:407] Connection header received
0809 12:17:44.737510 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:44.737515 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:44.737586 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:44.737698 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:44.738592 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:44.739467 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:44.741054 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:44.741664 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:44.741764 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:44.745581 (+ 3817us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.193:42829: BlockingRecv error: recv got EOF from 127.19.183.193:42829 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 09 12:17:45 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:45 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:45.250370 1107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:45.240971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.194:59691)
0809 12:17:45.241574 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:45.241590 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:45.241768 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:45.242137 (+ 369us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:45.242143 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:45.242633 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:45.242870 (+ 237us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:45.242880 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:45.243917 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:45.243923 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:45.244869 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:45.244880 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:45.245672 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:45.245679 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:45.245868 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:45.246743 (+ 875us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:45.246768 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:45.250055 (+ 3287us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":499,"thread_start_us":287,"threads_started":1}
W20260809 12:17:45.250720 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:45.241039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:59691 (local address 127.19.183.254:35275)
0809 12:17:45.241301 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:45.241309 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:45.241699 (+ 390us) server_negotiation.cc:407] Connection header received
0809 12:17:45.241911 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:45.241916 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:45.241990 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:45.242097 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:45.243018 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:45.243761 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:45.245024 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:45.245495 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:45.245634 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:45.250322 (+ 4688us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.194:59691: BlockingRecv error: recv got EOF from 127.19.183.194:59691 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 09 12:17:45 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:45 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:45.712232 1108 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:45.703125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.195:50219)
0809 12:17:45.703828 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:45.703851 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:45.704022 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:45.704503 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:45.704508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:45.705141 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:45.705390 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:45.705405 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:45.706489 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:45.706493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:45.707450 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:45.707463 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:45.708185 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:45.708190 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:45.708304 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:45.708981 (+ 677us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:45.709000 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:45.711961 (+ 2961us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":561,"thread_start_us":286,"threads_started":1}
W20260809 12:17:45.712610 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:45.703296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:50219 (local address 127.19.183.254:35275)
0809 12:17:45.703572 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:45.703582 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:45.703941 (+ 359us) server_negotiation.cc:407] Connection header received
0809 12:17:45.704228 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:45.704234 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:45.704326 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:45.704418 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:45.705565 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:45.706342 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:45.707604 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:45.708029 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:45.708138 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:45.712259 (+ 4121us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.195:50219: BlockingRecv error: recv got EOF from 127.19.183.195:50219 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 09 12:17:45 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:45 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:45.755959 1109 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:45.746575 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.193:42885)
0809 12:17:45.747250 (+ 675us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:45.747267 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:45.747394 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:45.747730 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:45.747733 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:45.748189 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:45.748423 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:45.748433 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:45.749754 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:45.749759 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:45.750771 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:45.750789 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:45.751696 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:45.751705 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:45.751873 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:45.752501 (+ 628us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:45.752520 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:45.755719 (+ 3199us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":388,"threads_started":1}
W20260809 12:17:45.756330 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:45.746936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:42885 (local address 127.19.183.254:35275)
0809 12:17:45.747151 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:45.747160 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:45.747351 (+ 191us) server_negotiation.cc:407] Connection header received
0809 12:17:45.747508 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:45.747512 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:45.747578 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:45.747666 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:45.748587 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:45.749587 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:45.750955 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:45.751480 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:45.751627 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:45.755964 (+ 4337us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.193:42885: BlockingRecv error: recv got EOF from 127.19.183.193:42885 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 12:17:46 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:46 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:46.265126 1112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:46.251860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.194:51005)
0809 12:17:46.252575 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:46.252597 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:46.252774 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:46.253302 (+ 528us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:46.253306 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:46.253754 (+ 448us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:46.253971 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:46.253981 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:46.255505 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:46.255510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:46.256467 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:46.256480 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:46.257248 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:46.257252 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:46.257365 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:46.261784 (+ 4419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:46.261805 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:46.264860 (+ 3055us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":375,"threads_started":1}
W20260809 12:17:46.265406 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:46.252022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:51005 (local address 127.19.183.254:35275)
0809 12:17:46.252302 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:46.252311 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:46.252716 (+ 405us) server_negotiation.cc:407] Connection header received
0809 12:17:46.252920 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:46.252924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:46.252994 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:46.253287 (+ 293us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:46.254134 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:46.255356 (+ 1222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:46.256610 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:46.257066 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:46.257166 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:46.265184 (+ 8018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.194:51005: BlockingRecv error: recv got EOF from 127.19.183.194:51005 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 09 12:17:46 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:46 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:46.723029 1113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:46.713411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.195:52495)
0809 12:17:46.714221 (+ 810us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:46.714239 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:46.714404 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:46.714811 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:46.714816 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:46.715281 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:46.715443 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:46.715454 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:46.716575 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:46.716581 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:46.717573 (+ 992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:46.717588 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:46.718391 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:46.718397 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:46.718581 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:46.719303 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:46.719324 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:46.722794 (+ 3470us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":518,"threads_started":1}
W20260809 12:17:46.723279 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:46.713506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:52495 (local address 127.19.183.254:35275)
0809 12:17:46.713725 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:46.713730 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:46.714338 (+ 608us) server_negotiation.cc:407] Connection header received
0809 12:17:46.714546 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:46.714555 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:46.714635 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:46.714772 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:46.715595 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:46.716414 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:46.717739 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:46.718202 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:46.718304 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:46.722993 (+ 4689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.195:52495: BlockingRecv error: recv got EOF from 127.19.183.195:52495 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 09 12:17:46 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:46 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:46.765472 1114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:46.756942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.193:40371)
0809 12:17:46.757482 (+ 540us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:46.757499 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:46.757646 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:46.757968 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:46.757971 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:46.758363 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:46.758534 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:46.758543 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:46.759543 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:46.759547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:46.760518 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:46.760529 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:46.761222 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:46.761227 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:46.761349 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:46.762027 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:46.762049 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:46.765219 (+ 3170us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":444,"thread_start_us":309,"threads_started":1}
W20260809 12:17:46.765762 689 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:46.757111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:40371 (local address 127.19.183.254:35275)
0809 12:17:46.757397 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:46.757404 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:46.757586 (+ 182us) server_negotiation.cc:407] Connection header received
0809 12:17:46.757765 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:46.757769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:46.757836 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:46.757919 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:46.758680 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:46.759405 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:46.760648 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:46.761087 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:46.761181 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:46.765466 (+ 4285us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.193:40371: BlockingRecv error: recv got EOF from 127.19.183.193:40371 (error 108)
Metrics: {"server-negotiator.queue_time_us":99}
Aug 09 12:17:47 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:47 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:47.276062 1115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:47.266370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.194:49081)
0809 12:17:47.266988 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:47.267014 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:47.267176 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:47.267691 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:47.267696 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:47.268169 (+ 473us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:47.268364 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:47.268374 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:47.269503 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:47.269509 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:47.270489 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:47.270501 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:47.271332 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:47.271338 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:47.271528 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:47.272209 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:47.272227 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:47.275799 (+ 3572us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":320,"threads_started":1}
W20260809 12:17:47.276506 1116 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:47.266508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:49081 (local address 127.19.183.254:35275)
0809 12:17:47.267361 (+ 853us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:47.267366 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:47.267381 (+ 15us) server_negotiation.cc:407] Connection header received
0809 12:17:47.267433 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:47.267436 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:47.267507 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:47.267630 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:47.268543 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:47.269356 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:47.270652 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:47.271167 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:47.271307 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:47.276018 (+ 4711us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.194:49081: BlockingRecv error: recv got EOF from 127.19.183.194:49081 (error 108)
Metrics: {"server-negotiator.queue_time_us":709,"thread_start_us":578,"threads_started":1}
Aug 09 12:17:47 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:47 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:47.732879 1118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:47.724102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.195:57121)
0809 12:17:47.724784 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:47.724801 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:47.724969 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:47.725380 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:47.725385 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:47.725824 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:47.725990 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:47.726010 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:47.727147 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:47.727155 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:47.728098 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:47.728111 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:47.728844 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:47.728850 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:47.729039 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:47.729632 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:47.729649 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:47.732590 (+ 2941us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":579,"thread_start_us":409,"threads_started":1}
W20260809 12:17:47.733091 1116 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:47.724309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:57121 (local address 127.19.183.254:35275)
0809 12:17:47.724659 (+ 350us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:47.724666 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:47.724895 (+ 229us) server_negotiation.cc:407] Connection header received
0809 12:17:47.725137 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:47.725147 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:47.725215 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:47.725323 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:47.726159 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:47.726994 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:47.728251 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:47.728681 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:47.728824 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:47.732834 (+ 4010us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.195:57121: BlockingRecv error: recv got EOF from 127.19.183.195:57121 (error 108)
Metrics: {"server-negotiator.queue_time_us":193}
Aug 09 12:17:47 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:47 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:47.775460 1121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:47.766592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.193:40057)
0809 12:17:47.767147 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:47.767162 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:47.767323 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:47.767720 (+ 397us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:47.767725 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:47.768151 (+ 426us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:47.768297 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:47.768307 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:47.769369 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:47.769374 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:47.770305 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:47.770317 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:47.771216 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:47.771223 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:47.771391 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:47.772059 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:47.772120 (+ 61us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:47.775098 (+ 2978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":459,"thread_start_us":336,"threads_started":1}
W20260809 12:17:47.775625 1116 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:47.766727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:40057 (local address 127.19.183.254:35275)
0809 12:17:47.766950 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:47.766959 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:47.767254 (+ 295us) server_negotiation.cc:407] Connection header received
0809 12:17:47.767475 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:47.767480 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:47.767556 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:47.767663 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:47.768448 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:47.769200 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:47.770448 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:47.770965 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:47.771196 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:47.775325 (+ 4129us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.193:40057: BlockingRecv error: recv got EOF from 127.19.183.193:40057 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 09 12:17:48 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:48 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:48.285393 1122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:48.277157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.194:55125)
0809 12:17:48.277728 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:48.277745 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:48.277858 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:48.278171 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:48.278175 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:48.278576 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:48.278822 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:48.278834 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:48.279822 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:48.279827 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:48.280777 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:48.280789 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:48.281465 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:48.281469 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:48.281583 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:48.282200 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:48.282218 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:48.285151 (+ 2933us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":307,"threads_started":1}
W20260809 12:17:48.285653 1124 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:48.277292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:55125 (local address 127.19.183.254:35275)
0809 12:17:48.277792 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:48.277796 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:48.277815 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:17:48.277952 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:48.277956 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:48.278026 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:48.278123 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:48.278947 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:48.279681 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:48.280909 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:48.281329 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:48.281422 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:48.285409 (+ 3987us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.194:55125: BlockingRecv error: recv got EOF from 127.19.183.194:55125 (error 108)
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":284,"threads_started":1}
Aug 09 12:17:48 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:48 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:48.742671 1125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:48.733828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.195:56609)
0809 12:17:48.734511 (+ 683us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:48.734528 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:48.734649 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:48.735025 (+ 376us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:48.735028 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:48.735417 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:48.735567 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:48.735575 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:48.736646 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:48.736650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:48.737596 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:48.737608 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:48.738289 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:48.738293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:48.738408 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:48.739233 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:48.739257 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:48.742326 (+ 3069us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":441,"threads_started":1}
W20260809 12:17:48.742929 1124 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:48.734018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:56609 (local address 127.19.183.254:35275)
0809 12:17:48.734248 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:48.734253 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:48.734630 (+ 377us) server_negotiation.cc:407] Connection header received
0809 12:17:48.734807 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:48.734811 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:48.734886 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:48.734974 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:48.735716 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:48.736474 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:48.737729 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:48.738151 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:48.738247 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:48.742622 (+ 4375us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.195:56609: BlockingRecv error: recv got EOF from 127.19.183.195:56609 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
Aug 09 12:17:48 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:48 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:48.785637 1126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:48.776512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.193:49643)
0809 12:17:48.777086 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:48.777106 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:48.777283 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:48.777662 (+ 379us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:48.777669 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:48.778170 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:48.778356 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:48.778367 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:48.779866 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:48.779872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:48.781108 (+ 1236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:48.781123 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:48.781816 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:48.781821 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:48.781945 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:48.782537 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:48.782555 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:48.785369 (+ 2814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":472,"thread_start_us":316,"threads_started":1}
W20260809 12:17:48.785851 1124 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:48.776650 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:49643 (local address 127.19.183.254:35275)
0809 12:17:48.776905 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:48.776914 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:48.777206 (+ 292us) server_negotiation.cc:407] Connection header received
0809 12:17:48.777427 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:48.777433 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:48.777507 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:48.777615 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:48.778509 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:48.779668 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:48.781241 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:48.781676 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:48.781775 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:48.785585 (+ 3810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.193:49643: BlockingRecv error: recv got EOF from 127.19.183.193:49643 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 09 12:17:49 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:49 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:49.295416 1127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:49.286673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.194:36743)
0809 12:17:49.287350 (+ 677us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:49.287372 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:49.287515 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:49.287957 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:49.287962 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:49.288528 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:49.288800 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:49.288814 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:49.289857 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:49.289860 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:49.290827 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:49.290839 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:49.291526 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:49.291531 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:49.291655 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:49.292263 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:49.292280 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:49.295171 (+ 2891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":511,"thread_start_us":288,"threads_started":1}
W20260809 12:17:49.295670 1128 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:49.286810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:36743 (local address 127.19.183.254:35275)
0809 12:17:49.287527 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:49.287533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:49.287554 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:17:49.287665 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:49.287669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:49.287773 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:49.287893 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:49.288975 (+ 1082us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:49.289739 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:49.290955 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:49.291387 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:49.291487 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:49.295353 (+ 3866us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.194:36743: BlockingRecv error: recv got EOF from 127.19.183.194:36743 (error 108)
Metrics: {"server-negotiator.queue_time_us":599,"thread_start_us":480,"threads_started":1}
Aug 09 12:17:49 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:49 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:49.753507 1129 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:49.743887 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.195:57979)
0809 12:17:49.744484 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:49.744501 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:49.744680 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:49.745052 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:49.745057 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:49.745535 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:49.745791 (+ 256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:49.745802 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:49.747031 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:49.747036 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:49.748126 (+ 1090us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:49.748141 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:49.749243 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:49.749248 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:49.749382 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:49.749981 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:49.749999 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:49.753128 (+ 3129us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":327,"threads_started":1}
W20260809 12:17:49.753616 1128 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:49.744084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:57979 (local address 127.19.183.254:35275)
0809 12:17:49.744345 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:49.744351 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:49.744625 (+ 274us) server_negotiation.cc:407] Connection header received
0809 12:17:49.744834 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:49.744839 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:49.744907 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:49.745016 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:49.745967 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:49.746878 (+ 911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:49.748313 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:49.749038 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:49.749155 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:49.753364 (+ 4209us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.195:57979: BlockingRecv error: recv got EOF from 127.19.183.195:57979 (error 108)
Metrics: {"server-negotiator.queue_time_us":92}
Aug 09 12:17:49 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:49 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:49.795287 1130 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:49.786752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.193:60367)
0809 12:17:49.787320 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:49.787337 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:49.787461 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:49.787796 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:49.787799 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:49.788192 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:49.788350 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:49.788358 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:49.789399 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:49.789404 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:49.790337 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:49.790350 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:49.791083 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:49.791087 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:49.791230 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:49.791853 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:49.791871 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:49.794939 (+ 3068us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":326,"threads_started":1}
W20260809 12:17:49.795482 1128 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:49.786835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:60367 (local address 127.19.183.254:35275)
0809 12:17:49.787070 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:49.787075 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:49.787449 (+ 374us) server_negotiation.cc:407] Connection header received
0809 12:17:49.787584 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:49.787588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:49.787655 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:49.787750 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:49.788478 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:49.789266 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:49.790485 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:49.790954 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:49.791080 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:49.795200 (+ 4120us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.193:60367: BlockingRecv error: recv got EOF from 127.19.183.193:60367 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 09 12:17:50 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:50 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:50.305037 1131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:50.296467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.194:36403)
0809 12:17:50.297525 (+ 1058us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:50.297546 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:50.297696 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:50.298042 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:50.298045 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:50.298462 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:50.298625 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:50.298635 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:50.299644 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:50.299648 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:50.300559 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:50.300570 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:50.301269 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:50.301273 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:50.301400 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:50.301983 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:50.302000 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:50.304806 (+ 2806us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":915,"thread_start_us":803,"threads_started":1}
W20260809 12:17:50.305270 1133 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:50.296516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:36403 (local address 127.19.183.254:35275)
0809 12:17:50.297178 (+ 662us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:50.297183 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:50.297650 (+ 467us) server_negotiation.cc:407] Connection header received
0809 12:17:50.297816 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:50.297819 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:50.297895 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:50.297989 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:50.298773 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:50.299517 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:50.300689 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:50.301141 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:50.301234 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:50.305013 (+ 3779us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.194:36403: BlockingRecv error: recv got EOF from 127.19.183.194:36403 (error 108)
Metrics: {"server-negotiator.queue_time_us":542,"thread_start_us":435,"threads_started":1}
Aug 09 12:17:50 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:50 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:50.763073 1134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:50.754433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.195:50085)
0809 12:17:50.755120 (+ 687us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:50.755138 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:50.755257 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:50.755571 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:50.755574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:50.756023 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:50.756197 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:50.756206 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:50.757236 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:50.757240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:50.758302 (+ 1062us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:50.758322 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:50.759156 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:50.759162 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:50.759335 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:50.759984 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:50.760002 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:50.762845 (+ 2843us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":563,"thread_start_us":437,"threads_started":1}
W20260809 12:17:50.763309 1133 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:50.754513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:50085 (local address 127.19.183.254:35275)
0809 12:17:50.754691 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:50.754695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:50.755217 (+ 522us) server_negotiation.cc:407] Connection header received
0809 12:17:50.755357 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:50.755361 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:50.755426 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:50.755513 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:50.756343 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:50.757108 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:50.758471 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:50.758975 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:50.759112 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:50.763042 (+ 3930us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.195:50085: BlockingRecv error: recv got EOF from 127.19.183.195:50085 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 09 12:17:50 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:50 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:50.804421 1135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:50.796303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.193:32997)
0809 12:17:50.796864 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:50.796880 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:50.797000 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:50.797335 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:50.797338 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:50.797727 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:50.797885 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:50.797894 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:50.798961 (+ 1067us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:50.798966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:50.799900 (+ 934us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:50.799912 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:50.800585 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:50.800590 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:50.800708 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:50.801345 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:50.801363 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:50.804177 (+ 2814us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":292,"threads_started":1}
W20260809 12:17:50.804572 1133 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:50.796410 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:32997 (local address 127.19.183.254:35275)
0809 12:17:50.796587 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:50.796592 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:50.796959 (+ 367us) server_negotiation.cc:407] Connection header received
0809 12:17:50.797103 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:50.797106 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:50.797188 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:50.797270 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:50.798022 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:50.798840 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:50.800023 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:50.800431 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:50.800522 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:50.804358 (+ 3836us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.193:32997: BlockingRecv error: recv got EOF from 127.19.183.193:32997 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 12:17:51 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:51 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:51.318212 1136 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:51.306023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.194:33987)
0809 12:17:51.306603 (+ 580us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:51.306621 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:51.306792 (+ 171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:51.307149 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:51.307152 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:51.307538 (+ 386us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:51.307702 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:51.307711 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:51.308649 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:51.308653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:51.309533 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:51.309572 (+ 39us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:51.310253 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:51.310257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:51.310368 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:51.315053 (+ 4685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:51.315070 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:51.317964 (+ 2894us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":259,"threads_started":1}
W20260809 12:17:51.318431 1140 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:51.306171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:33987 (local address 127.19.183.254:35275)
0809 12:17:51.306631 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:51.306636 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:51.306694 (+ 58us) server_negotiation.cc:407] Connection header received
0809 12:17:51.306920 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:51.306924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:51.306999 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:51.307091 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:51.307830 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:51.308520 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:51.309678 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:51.310096 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:51.310191 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:51.318182 (+ 7991us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.194:33987: BlockingRecv error: recv got EOF from 127.19.183.194:33987 (error 108)
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":244,"threads_started":1}
Aug 09 12:17:51 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:51 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:51.772815 1142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:51.763961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.195:41333)
0809 12:17:51.764489 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:51.764506 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:51.764674 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:51.765024 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:51.765046 (+ 22us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:51.765485 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:51.765663 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:51.765674 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:51.766698 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:51.766728 (+ 30us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:51.767631 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:51.767642 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:51.768368 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:51.768373 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:51.768546 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:51.769175 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:51.769194 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:51.772450 (+ 3256us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":431,"thread_start_us":277,"threads_started":1}
W20260809 12:17:51.772939 1140 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:51.764014 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:41333 (local address 127.19.183.254:35275)
0809 12:17:51.764243 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:51.764252 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:51.764597 (+ 345us) server_negotiation.cc:407] Connection header received
0809 12:17:51.764801 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:51.764810 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:51.764886 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:51.764985 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:51.765815 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:51.766544 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:51.767774 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:51.768222 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:51.768355 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:51.772691 (+ 4336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.195:41333: BlockingRecv error: recv got EOF from 127.19.183.195:41333 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 12:17:51 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:51 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:51.814327 1143 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:51.805568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.193:33937)
0809 12:17:51.806062 (+ 494us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:51.806082 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:51.806220 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:51.806552 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:51.806556 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:51.807037 (+ 481us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:51.807310 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:51.807327 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:51.808703 (+ 1376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:51.808708 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:51.809597 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:51.809608 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:51.810225 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:51.810229 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:51.810343 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:51.811036 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:51.811059 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:51.813942 (+ 2883us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":262,"threads_started":1}
W20260809 12:17:51.814412 1140 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:51.805712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:33937 (local address 127.19.183.254:35275)
0809 12:17:51.805962 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:51.805967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:51.806186 (+ 219us) server_negotiation.cc:407] Connection header received
0809 12:17:51.806324 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:51.806327 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:51.806403 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:51.806485 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:51.807476 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:51.808571 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:51.809716 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:51.810106 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:51.810207 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:51.814133 (+ 3926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.193:33937: BlockingRecv error: recv got EOF from 127.19.183.193:33937 (error 108)
Metrics: {"server-negotiator.queue_time_us":137}
Aug 09 12:17:52 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:52 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:52.327658 1144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:52.319164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.194:50287)
0809 12:17:52.319660 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:52.319676 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:52.319784 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:52.320126 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:52.320129 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:52.320497 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:52.320688 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:52.320697 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:52.321749 (+ 1052us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:52.321753 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:52.322651 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:52.322662 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:52.323450 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:52.323455 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:52.323582 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:52.324265 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:52.324283 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:52.327297 (+ 3014us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":262,"threads_started":1}
W20260809 12:17:52.327919 1145 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:52.319306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:50287 (local address 127.19.183.254:35275)
0809 12:17:52.319754 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:52.319759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:52.319776 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:17:52.319898 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:52.319902 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:52.319976 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:52.320082 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:52.320852 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:52.321617 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:52.322855 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:52.323315 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:52.323420 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:52.327525 (+ 4105us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.194:50287: BlockingRecv error: recv got EOF from 127.19.183.194:50287 (error 108)
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":253,"threads_started":1}
Aug 09 12:17:52 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:52 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.195@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:52.781730 1146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:52.773747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.195:51469)
0809 12:17:52.774347 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:52.774364 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:52.774483 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:52.774918 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:52.774926 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:52.775300 (+ 374us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:52.775482 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:52.775491 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:52.776492 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:52.776496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:52.777392 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:52.777403 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:52.778044 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:52.778048 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:52.778188 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:52.778824 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:52.778842 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:52.781508 (+ 2666us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":453,"thread_start_us":328,"threads_started":1}
W20260809 12:17:52.781965 1145 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:52.773819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:51469 (local address 127.19.183.254:35275)
0809 12:17:52.774025 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:52.774031 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:52.774454 (+ 423us) server_negotiation.cc:407] Connection header received
0809 12:17:52.774615 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:52.774624 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:52.774706 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:52.774827 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:52.775616 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:52.776367 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:52.777513 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:52.777910 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:52.778004 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:52.781709 (+ 3705us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.195:51469: BlockingRecv error: recv got EOF from 127.19.183.195:51469 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 09 12:17:52 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:52 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.193@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:52.824282 1148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:52.815725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.193:38361)
0809 12:17:52.816334 (+ 609us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:52.816351 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:52.816468 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:52.816804 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:52.816807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:52.817228 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:52.817406 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:52.817415 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:52.818351 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:52.818355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:52.819378 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:52.819391 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:52.820023 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:52.820027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:52.820152 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:52.820820 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:52.820864 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:52.824004 (+ 3140us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":488,"thread_start_us":329,"threads_started":1}
W20260809 12:17:52.824579 1145 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:52.815765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:38361 (local address 127.19.183.254:35275)
0809 12:17:52.815950 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:52.815954 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:52.816428 (+ 474us) server_negotiation.cc:407] Connection header received
0809 12:17:52.816569 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:52.816573 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:52.816654 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:52.816735 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:52.817528 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:52.818226 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:52.819505 (+ 1279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:52.819904 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:52.819993 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:52.824185 (+ 4192us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.193:38361: BlockingRecv error: recv got EOF from 127.19.183.193:38361 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 12:17:53 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
Aug 09 12:17:53 dist-test-slave-39l8 krb5kdc[32065](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.19.183.194@KRBTEST.COM for oryx/127.19.183.254@KRBTEST.COM, Server not found in Kerberos database
I20260809 12:17:53.337714 1149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:53.328893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35275 (local address 127.19.183.194:58013)
0809 12:17:53.329423 (+ 530us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:53.329440 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:53.329545 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:53.330037 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:53.330040 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:53.330462 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:53.330633 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:53.330642 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:53.331982 (+ 1340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:53.331999 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:53.332982 (+ 983us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:53.332995 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:53.333755 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:53.333761 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:53.333950 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:53.334534 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:17:53.334552 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:53.337342 (+ 2790us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.19.183.254:35275: Server oryx/127.19.183.254@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":281,"threads_started":1}
W20260809 12:17:53.337898 1150 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:53.328965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:58013 (local address 127.19.183.254:35275)
0809 12:17:53.329755 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:53.329760 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:53.329775 (+ 15us) server_negotiation.cc:407] Connection header received
0809 12:17:53.329820 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:53.329824 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:53.329890 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:53.329982 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:53.330816 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:53.331825 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:53.333138 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:53.333595 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:53.333741 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:53.337584 (+ 3843us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.19.183.194:58013: BlockingRecv error: recv got EOF from 127.19.183.194:58013 (error 108)
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":575,"threads_started":1}
I20260809 12:17:53.612615 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 461
W20260809 12:17:53.625519 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 461: Not authorized: Client connection negotiation failed: client connection to 127.19.183.193:36689: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.19.183.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 12:17:53.625474 1155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:53.613567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37074 (local address 127.19.183.193:36689)
0809 12:17:53.614194 (+ 627us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:53.614202 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:53.614229 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:17:53.614289 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:53.614296 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:53.614514 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:53.614690 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:53.616167 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:53.617224 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:53.618417 (+ 1193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:53.619116 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:53.619225 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:53.621642 (+ 2417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:53.621651 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:53.621665 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:53.621703 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:53.624839 (+ 3136us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.19.183.193@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 12:17:53.625163 (+ 324us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37074: Request ticket server kudu/127.19.183.193@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":441,"thread_start_us":315,"threads_started":1}
W20260809 12:17:53.625700 1155 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37074: Request ticket server kudu/127.19.183.193@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 12:17:53.625735 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 461
I20260809 12:17:53.662087 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 688
I20260809 12:17:53.674402 1156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:53.663113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39886 (local address 127.19.183.194:39137)
0809 12:17:53.663723 (+ 610us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:53.663730 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:53.663763 (+ 33us) server_negotiation.cc:407] Connection header received
0809 12:17:53.663830 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:53.663837 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:53.664059 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:53.664225 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:53.665463 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:53.666565 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:53.667854 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:53.668396 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:53.668495 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:53.671100 (+ 2605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:53.671109 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:53.671134 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:53.671172 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:53.673909 (+ 2737us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.19.183.194@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 12:17:53.674168 (+ 259us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:39886: Request ticket server kudu/127.19.183.194@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":276,"threads_started":1}
W20260809 12:17:53.674543 1156 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:39886: Request ticket server kudu/127.19.183.194@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 12:17:53.674541 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 688: Not authorized: Client connection negotiation failed: client connection to 127.19.183.194:39137: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.19.183.194@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 12:17:53.674784 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 688
I20260809 12:17:53.710170 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 888
I20260809 12:17:53.722947 1158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:53.711298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45814 (local address 127.19.183.195:42767)
0809 12:17:53.711869 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:53.711877 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:53.711902 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:17:53.711967 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:53.711974 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:53.712228 (+ 254us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:53.712425 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:53.713829 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:53.714933 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:53.716186 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:53.716721 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:53.716865 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:53.719465 (+ 2600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:53.719475 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:53.719488 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:17:53.719526 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:53.722157 (+ 2631us) server_negotiation.cc:394] Sending RPC error: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.19.183.195@KRBTEST.COM not found in keytab (ticket kvno 2)
0809 12:17:53.722481 (+ 324us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45814: Request ticket server kudu/127.19.183.195@KRBTEST.COM not found in keytab (ticket kvno 2)
Metrics: {"server-negotiator.queue_time_us":419,"thread_start_us":302,"threads_started":1}
W20260809 12:17:53.722944 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 888: Not authorized: Client connection negotiation failed: client connection to 127.19.183.195:42767: FATAL_UNAUTHORIZED: Not authorized: Request ticket server kudu/127.19.183.195@KRBTEST.COM not found in keytab (ticket kvno 2)
W20260809 12:17:53.723060 1158 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45814: Request ticket server kudu/127.19.183.195@KRBTEST.COM not found in keytab (ticket kvno 2)
I20260809 12:17:53.723109 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 888
I20260809 12:17:53.760815 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 356
I20260809 12:17:53.762039 452 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:53.881999 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 356
2026-08-09T12:17:53Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (22669 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T12:17:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:17:53Z Disabled control of system clock
I20260809 12:17:53.983031 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:35699
--webserver_interface=127.19.183.254
--webserver_port=0
--builtin_ntp_servers=127.19.183.212:33087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:35699
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 12:17:54.344525 1163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:54.344807 1163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:54.344975 1163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:54.352035 1163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:17:54.352159 1163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:54.352231 1163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:54.352332 1163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:17:54.352416 1163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:17:54.361429 1163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:33087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:35699
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:35699
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:54.362793 1163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:54.364596 1163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:54.379222 1172 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:54.379339 1174 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:54.379303 1177 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:54.380295 1163 server_base.cc:1034] running on GCE node
I20260809 12:17:54.381131 1163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:54.382531 1163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:54.383744 1163 hybrid_clock.cc:648] HybridClock initialized: now 1786277874383690 us; error 48 us; skew 500 ppm
I20260809 12:17:54.384063 1163 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:17:54.386271 1163 webserver.cc:460] Webserver started at http://127.19.183.254:45875/ using document root <none> and password file <none>
I20260809 12:17:54.387027 1163 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:54.387118 1163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:54.387411 1163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:54.390314 1163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "4199979c6a9b4f79ad5c5c21f5d2abb2"
format_stamp: "Formatted at 2026-08-09 12:17:54 on dist-test-slave-39l8"
server_key: "c5309ab041472ca9aa76f4af7c614b00"
server_key_iv: "7a97643fb24f5663db9b757d4644c808"
server_key_version: "encryptionkey@0"
I20260809 12:17:54.391212 1163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "4199979c6a9b4f79ad5c5c21f5d2abb2"
format_stamp: "Formatted at 2026-08-09 12:17:54 on dist-test-slave-39l8"
server_key: "c5309ab041472ca9aa76f4af7c614b00"
server_key_iv: "7a97643fb24f5663db9b757d4644c808"
server_key_version: "encryptionkey@0"
I20260809 12:17:54.396404 1163 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:17:54.399802 1193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:54.401201 1163 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:17:54.401366 1163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "4199979c6a9b4f79ad5c5c21f5d2abb2"
format_stamp: "Formatted at 2026-08-09 12:17:54 on dist-test-slave-39l8"
server_key: "c5309ab041472ca9aa76f4af7c614b00"
server_key_iv: "7a97643fb24f5663db9b757d4644c808"
server_key_version: "encryptionkey@0"
I20260809 12:17:54.401607 1163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:54.423949 1163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:54.424842 1163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:54.425160 1163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:54.444388 1163 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:35699
I20260809 12:17:54.444409 1267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:35699 every 8 connection(s)
I20260809 12:17:54.445808 1163 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:17:54.449683 1271 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:54.452404 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 1163
I20260809 12:17:54.452826 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:17:54.453917 20191 external_mini_cluster.cc:1428] Setting key ef1ab09a6b6d0683805cde85564b612a
I20260809 12:17:54.460835 1271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4199979c6a9b4f79ad5c5c21f5d2abb2: Bootstrap starting.
I20260809 12:17:54.464403 1271 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4199979c6a9b4f79ad5c5c21f5d2abb2: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:54.465583 1271 log.cc:824] T 00000000000000000000000000000000 P 4199979c6a9b4f79ad5c5c21f5d2abb2: Log is configured to *not* fsync() on all Append() calls
I20260809 12:17:54.469056 1271 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4199979c6a9b4f79ad5c5c21f5d2abb2: No bootstrap required, opened a new log
I20260809 12:17:54.469383 1276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:54.456245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53306 (local address 127.19.183.254:35699)
0809 12:17:54.457303 (+ 1058us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:54.457320 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:54.457369 (+ 49us) server_negotiation.cc:407] Connection header received
0809 12:17:54.459580 (+ 2211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:54.459604 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:54.460086 (+ 482us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:54.460425 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:54.461167 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:54.462629 (+ 1462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:54.464700 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:54.465407 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:54.465540 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:54.466884 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:54.466899 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:54.466921 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:54.467616 (+ 695us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:54.467692 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:54.467714 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:54.467803 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:54.468061 (+ 258us) server_negotiation.cc:299] Negotiation successful
0809 12:17:54.468416 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":395,"threads_started":1}
I20260809 12:17:54.477190 1271 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4199979c6a9b4f79ad5c5c21f5d2abb2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4199979c6a9b4f79ad5c5c21f5d2abb2" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35699 } }
I20260809 12:17:54.477596 1271 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4199979c6a9b4f79ad5c5c21f5d2abb2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:54.477716 1271 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4199979c6a9b4f79ad5c5c21f5d2abb2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4199979c6a9b4f79ad5c5c21f5d2abb2, State: Initialized, Role: FOLLOWER
I20260809 12:17:54.478232 1271 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4199979c6a9b4f79ad5c5c21f5d2abb2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4199979c6a9b4f79ad5c5c21f5d2abb2" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35699 } }
I20260809 12:17:54.478435 1271 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4199979c6a9b4f79ad5c5c21f5d2abb2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:17:54.478541 1271 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4199979c6a9b4f79ad5c5c21f5d2abb2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:17:54.478667 1271 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4199979c6a9b4f79ad5c5c21f5d2abb2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:54.481108 1271 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4199979c6a9b4f79ad5c5c21f5d2abb2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4199979c6a9b4f79ad5c5c21f5d2abb2" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35699 } }
I20260809 12:17:54.481711 1271 leader_election.cc:302] T 00000000000000000000000000000000 P 4199979c6a9b4f79ad5c5c21f5d2abb2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4199979c6a9b4f79ad5c5c21f5d2abb2; no voters:
I20260809 12:17:54.482329 1271 leader_election.cc:288] T 00000000000000000000000000000000 P 4199979c6a9b4f79ad5c5c21f5d2abb2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:17:54.482474 1279 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4199979c6a9b4f79ad5c5c21f5d2abb2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:17:54.483243 1279 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4199979c6a9b4f79ad5c5c21f5d2abb2 [term 1 LEADER]: Becoming Leader. State: Replica: 4199979c6a9b4f79ad5c5c21f5d2abb2, State: Running, Role: LEADER
I20260809 12:17:54.483835 1279 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4199979c6a9b4f79ad5c5c21f5d2abb2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4199979c6a9b4f79ad5c5c21f5d2abb2" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35699 } }
I20260809 12:17:54.484488 1271 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4199979c6a9b4f79ad5c5c21f5d2abb2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:17:54.487052 1281 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4199979c6a9b4f79ad5c5c21f5d2abb2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4199979c6a9b4f79ad5c5c21f5d2abb2. Latest consensus state: current_term: 1 leader_uuid: "4199979c6a9b4f79ad5c5c21f5d2abb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4199979c6a9b4f79ad5c5c21f5d2abb2" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35699 } } }
I20260809 12:17:54.487174 1280 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4199979c6a9b4f79ad5c5c21f5d2abb2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4199979c6a9b4f79ad5c5c21f5d2abb2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4199979c6a9b4f79ad5c5c21f5d2abb2" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35699 } } }
I20260809 12:17:54.487557 1281 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4199979c6a9b4f79ad5c5c21f5d2abb2 [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:54.487655 1280 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4199979c6a9b4f79ad5c5c21f5d2abb2 [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:54.490099 1285 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:17:54.495772 1285 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:17:54.505033 1285 catalog_manager.cc:1347] Generated new cluster ID: 0fa68d41498748c290b8eb8fe4a4a5d7
I20260809 12:17:54.505167 1285 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:17:54.522596 1285 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:17:54.523881 1285 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:17:54.531540 1285 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4199979c6a9b4f79ad5c5c21f5d2abb2: Generated new TSK 0
I20260809 12:17:54.532332 1285 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:17:54.542855 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35699
--builtin_ntp_servers=127.19.183.212:33087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 12:17:54.916821 1309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:54.917131 1309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:54.917286 1309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:54.924571 1309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:54.924671 1309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:54.924837 1309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:17:54.934067 1309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:33087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:54.935448 1309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:54.937425 1309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:54.950455 1316 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:54.950502 1321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:54.950524 1319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:54.951436 1309 server_base.cc:1034] running on GCE node
I20260809 12:17:54.952085 1309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:54.953337 1309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:54.954566 1309 hybrid_clock.cc:648] HybridClock initialized: now 1786277874954494 us; error 58 us; skew 500 ppm
I20260809 12:17:54.955040 1309 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:17:54.957490 1309 webserver.cc:460] Webserver started at http://127.19.183.193:45387/ using document root <none> and password file <none>
I20260809 12:17:54.958259 1309 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:54.958390 1309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:54.958806 1309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:54.961668 1309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "24c12190c3574967938aecba6619b7f9"
format_stamp: "Formatted at 2026-08-09 12:17:54 on dist-test-slave-39l8"
server_key: "c01eaeccd61b687c44dbb1cb70efea3a"
server_key_iv: "9dcac4fc45bf3174df8d8073112c13ac"
server_key_version: "encryptionkey@0"
I20260809 12:17:54.962483 1309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "24c12190c3574967938aecba6619b7f9"
format_stamp: "Formatted at 2026-08-09 12:17:54 on dist-test-slave-39l8"
server_key: "c01eaeccd61b687c44dbb1cb70efea3a"
server_key_iv: "9dcac4fc45bf3174df8d8073112c13ac"
server_key_version: "encryptionkey@0"
I20260809 12:17:54.967914 1309 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:17:54.971712 1327 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:54.973061 1309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:17:54.973249 1309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "24c12190c3574967938aecba6619b7f9"
format_stamp: "Formatted at 2026-08-09 12:17:54 on dist-test-slave-39l8"
server_key: "c01eaeccd61b687c44dbb1cb70efea3a"
server_key_iv: "9dcac4fc45bf3174df8d8073112c13ac"
server_key_version: "encryptionkey@0"
I20260809 12:17:54.973486 1309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:54.997287 1309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:54.998189 1309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:54.998512 1309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:54.999622 1309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:55.001153 1309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:55.001243 1309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:55.001396 1309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:55.001492 1309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:55.036194 1309 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:36431
I20260809 12:17:55.036291 1498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:36431 every 8 connection(s)
I20260809 12:17:55.037578 1309 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:17:55.038640 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 1309
I20260809 12:17:55.038887 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:17:55.039734 20191 external_mini_cluster.cc:1428] Setting key ea3484e6fc3142566ef19be15ac5c010
I20260809 12:17:55.043141 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35699
--builtin_ntp_servers=127.19.183.212:33087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 12:17:55.054492 1504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:55.041754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:48283 (local address 127.19.183.254:35699)
0809 12:17:55.043100 (+ 1346us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:55.043105 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:55.043123 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:17:55.045738 (+ 2615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:55.045742 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:55.045809 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:55.045904 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:55.047399 (+ 1495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:55.048114 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:55.050679 (+ 2565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:55.051125 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:55.051227 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:55.053600 (+ 2373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:55.053605 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:55.053608 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:55.053628 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:55.053675 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:55.053687 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:55.053790 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:55.054028 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 12:17:55.054133 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1245,"thread_start_us":1151,"threads_started":1}
I20260809 12:17:55.055660 1502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:55.041264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35699 (local address 127.19.183.193:48283)
0809 12:17:55.042502 (+ 1238us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:55.042572 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:55.045434 (+ 2862us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:55.046185 (+ 751us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:55.046199 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:55.046333 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:55.047216 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:55.047246 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:55.048289 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:55.048297 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:55.050502 (+ 2205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:55.050523 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:55.051356 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:55.051366 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:55.051707 (+ 341us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:55.053134 (+ 1427us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:17:55.053138 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:55.053215 (+ 77us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:17:55.053347 (+ 132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:55.053792 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:55.053806 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:55.053816 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 12:17:55.054017 (+ 201us) client_negotiation.cc:240] Negotiation successful
0809 12:17:55.054436 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":574,"thread_start_us":541,"threads_started":1}
I20260809 12:17:55.057390 1499 heartbeater.cc:347] Connected to a master server at 127.19.183.254:35699
I20260809 12:17:55.057716 1499 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:55.058509 1499 heartbeater.cc:511] Master 127.19.183.254:35699 requested a full tablet report, sending...
I20260809 12:17:55.060624 1218 ts_manager.cc:194] Registered new tserver with Master: 24c12190c3574967938aecba6619b7f9 (127.19.183.193:36431)
I20260809 12:17:55.062150 1218 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.183.193:48283
W20260809 12:17:55.392069 1505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:55.392379 1505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:55.392535 1505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:55.399763 1505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:55.399893 1505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:55.400075 1505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:17:55.409171 1505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:33087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:55.410497 1505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:55.412480 1505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:55.425853 1517 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:55.425904 1514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:55.426054 1511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:55.426985 1505 server_base.cc:1034] running on GCE node
I20260809 12:17:55.427613 1505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:55.428570 1505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:55.429813 1505 hybrid_clock.cc:648] HybridClock initialized: now 1786277875429768 us; error 57 us; skew 500 ppm
I20260809 12:17:55.430145 1505 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:17:55.432476 1505 webserver.cc:460] Webserver started at http://127.19.183.194:36485/ using document root <none> and password file <none>
I20260809 12:17:55.433219 1505 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:55.433350 1505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:55.433712 1505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:55.436656 1505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "171ac5c9d3e846beb53eeaa1a777b4e7"
format_stamp: "Formatted at 2026-08-09 12:17:55 on dist-test-slave-39l8"
server_key: "4eb4ca9f44ba36ed1abea3d55f76885a"
server_key_iv: "bb111c27d99e82a1414b83655f8b4b62"
server_key_version: "encryptionkey@0"
I20260809 12:17:55.437466 1505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "171ac5c9d3e846beb53eeaa1a777b4e7"
format_stamp: "Formatted at 2026-08-09 12:17:55 on dist-test-slave-39l8"
server_key: "4eb4ca9f44ba36ed1abea3d55f76885a"
server_key_iv: "bb111c27d99e82a1414b83655f8b4b62"
server_key_version: "encryptionkey@0"
I20260809 12:17:55.442694 1505 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 12:17:55.446496 1526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:55.447970 1505 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:17:55.448149 1505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "171ac5c9d3e846beb53eeaa1a777b4e7"
format_stamp: "Formatted at 2026-08-09 12:17:55 on dist-test-slave-39l8"
server_key: "4eb4ca9f44ba36ed1abea3d55f76885a"
server_key_iv: "bb111c27d99e82a1414b83655f8b4b62"
server_key_version: "encryptionkey@0"
I20260809 12:17:55.448370 1505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:55.466436 1505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:55.467317 1505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:55.467623 1505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:55.468556 1505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:55.469995 1505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:55.470118 1505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:55.470242 1505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:55.470331 1505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:55.504575 1505 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:44317
I20260809 12:17:55.504666 1685 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:44317 every 8 connection(s)
I20260809 12:17:55.506196 1505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:17:55.516099 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 1505
I20260809 12:17:55.516325 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:17:55.516924 20191 external_mini_cluster.cc:1428] Setting key 649ee0b56e901cc7309489ff755ca270
I20260809 12:17:55.519085 1504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:55.508899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:60035 (local address 127.19.183.254:35699)
0809 12:17:55.509085 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:55.509089 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:55.510481 (+ 1392us) server_negotiation.cc:407] Connection header received
0809 12:17:55.512330 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:55.512335 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:55.512406 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:55.512498 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:55.513543 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:55.514261 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:55.515984 (+ 1723us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:55.516469 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:55.516567 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:55.518228 (+ 1661us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:55.518234 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:55.518239 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:55.518262 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:55.518302 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:55.518317 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:55.518412 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:55.518624 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 12:17:55.518742 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:17:55.520051 1688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:55.509454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35699 (local address 127.19.183.194:60035)
0809 12:17:55.510340 (+ 886us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:55.510384 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:55.512068 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:55.512730 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:55.512738 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:55.512822 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:55.513393 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:55.513410 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:55.514398 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:55.514402 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:55.515832 (+ 1430us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:55.515859 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:55.516704 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:55.516712 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:55.517012 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:55.517867 (+ 855us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:17:55.517871 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:55.517925 (+ 54us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:17:55.518019 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:55.518420 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:55.518429 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:55.518435 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 12:17:55.518572 (+ 137us) client_negotiation.cc:240] Negotiation successful
0809 12:17:55.519005 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":464,"thread_start_us":270,"threads_started":1}
I20260809 12:17:55.520114 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35699
--builtin_ntp_servers=127.19.183.212:33087
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 12:17:55.521381 1686 heartbeater.cc:347] Connected to a master server at 127.19.183.254:35699
I20260809 12:17:55.521710 1686 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:55.522657 1686 heartbeater.cc:511] Master 127.19.183.254:35699 requested a full tablet report, sending...
I20260809 12:17:55.524139 1218 ts_manager.cc:194] Registered new tserver with Master: 171ac5c9d3e846beb53eeaa1a777b4e7 (127.19.183.194:44317)
I20260809 12:17:55.524925 1218 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.183.194:60035
W20260809 12:17:55.866137 1689 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:55.866463 1689 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:55.866621 1689 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:55.873921 1689 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:55.874065 1689 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:55.874238 1689 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:17:55.883539 1689 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:33087
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35699
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:55.885004 1689 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:55.886919 1689 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:55.900034 1701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:55.899999 1702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:55.900223 1704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:55.900857 1689 server_base.cc:1034] running on GCE node
I20260809 12:17:55.901501 1689 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:55.902428 1689 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:55.903659 1689 hybrid_clock.cc:648] HybridClock initialized: now 1786277875903588 us; error 66 us; skew 500 ppm
I20260809 12:17:55.903990 1689 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:17:55.906374 1689 webserver.cc:460] Webserver started at http://127.19.183.195:40143/ using document root <none> and password file <none>
I20260809 12:17:55.907168 1689 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:55.907311 1689 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:55.907653 1689 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:55.910580 1689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "df2c53138c4545a284f7f34a6cd3c837"
format_stamp: "Formatted at 2026-08-09 12:17:55 on dist-test-slave-39l8"
server_key: "7d26801f3b7ea533f2793c63f8f08a98"
server_key_iv: "235158b21eaa6588a51886c811136396"
server_key_version: "encryptionkey@0"
I20260809 12:17:55.911490 1689 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "df2c53138c4545a284f7f34a6cd3c837"
format_stamp: "Formatted at 2026-08-09 12:17:55 on dist-test-slave-39l8"
server_key: "7d26801f3b7ea533f2793c63f8f08a98"
server_key_iv: "235158b21eaa6588a51886c811136396"
server_key_version: "encryptionkey@0"
I20260809 12:17:55.916906 1689 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:17:55.920380 1714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:55.921669 1689 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:17:55.921878 1689 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "df2c53138c4545a284f7f34a6cd3c837"
format_stamp: "Formatted at 2026-08-09 12:17:55 on dist-test-slave-39l8"
server_key: "7d26801f3b7ea533f2793c63f8f08a98"
server_key_iv: "235158b21eaa6588a51886c811136396"
server_key_version: "encryptionkey@0"
I20260809 12:17:55.922120 1689 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:55.936918 1689 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:55.937767 1689 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:55.938102 1689 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:55.939167 1689 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:55.940647 1689 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:55.940794 1689 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:55.940932 1689 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:55.941031 1689 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:55.977361 1689 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:42689
I20260809 12:17:55.977454 1869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:42689 every 8 connection(s)
I20260809 12:17:55.978752 1689 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:17:55.979837 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 1689
I20260809 12:17:55.980029 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:17:55.980587 20191 external_mini_cluster.cc:1428] Setting key 570caa3511548f19d8531649d2daa0b2
I20260809 12:17:55.992003 1504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:55.981671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:48185 (local address 127.19.183.254:35699)
0809 12:17:55.981885 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:55.981891 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:55.983629 (+ 1738us) server_negotiation.cc:407] Connection header received
0809 12:17:55.985508 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:55.985513 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:55.985605 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:55.985722 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:55.986794 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:55.987575 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:55.989115 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:55.989523 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:55.989672 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:55.991321 (+ 1649us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:55.991326 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:55.991329 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:55.991345 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:55.991397 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:55.991409 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:55.991485 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:55.991674 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 12:17:55.991749 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 12:17:55.992950 1873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:55.982309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35699 (local address 127.19.183.195:48185)
0809 12:17:55.983475 (+ 1166us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:55.983520 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:55.985233 (+ 1713us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:55.985955 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:55.985962 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:55.986057 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:55.986604 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:55.986621 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:55.987747 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:55.987758 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:55.988959 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:17:55.988971 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:17:55.989783 (+ 812us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:17:55.989791 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:17:55.990053 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:55.990908 (+ 855us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:17:55.990912 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:55.990985 (+ 73us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:17:55.991100 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:55.991489 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:55.991496 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:55.991502 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 12:17:55.991638 (+ 136us) client_negotiation.cc:240] Negotiation successful
0809 12:17:55.991904 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":639,"thread_start_us":421,"threads_started":1}
I20260809 12:17:55.994395 1871 heartbeater.cc:347] Connected to a master server at 127.19.183.254:35699
I20260809 12:17:55.994747 1871 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:55.995641 1871 heartbeater.cc:511] Master 127.19.183.254:35699 requested a full tablet report, sending...
I20260809 12:17:55.997058 1218 ts_manager.cc:194] Registered new tserver with Master: df2c53138c4545a284f7f34a6cd3c837 (127.19.183.195:42689)
I20260809 12:17:55.997821 1218 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.183.195:48185
I20260809 12:17:56.008324 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 12:17:56.015887 1504 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:56.013452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53310 (local address 127.19.183.254:35699)
0809 12:17:56.013697 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:56.013704 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:56.014583 (+ 879us) server_negotiation.cc:407] Connection header received
0809 12:17:56.014816 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:56.014823 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:56.014924 (+ 101us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:56.015034 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:56.015606 (+ 572us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53310: BlockingRecv error: recv got EOF from 127.0.0.1:53310 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 12:17:56.016459 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 1309
I20260809 12:17:56.025106 1884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:56.017559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45864 (local address 127.19.183.193:36431)
0809 12:17:56.018180 (+ 621us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:56.018189 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:56.018307 (+ 118us) server_negotiation.cc:407] Connection header received
0809 12:17:56.018460 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:56.018466 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:56.018673 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:56.018878 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:56.019524 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:56.020499 (+ 975us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:56.021941 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:56.022483 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:56.022587 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:56.023670 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:56.023679 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:56.023692 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:56.024296 (+ 604us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:56.024342 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:56.024362 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:56.024454 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:56.024588 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 12:17:56.024769 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":451,"thread_start_us":347,"threads_started":1}
I20260809 12:17:56.026463 1493 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:56.108319 1499 heartbeater.cc:503] Master 127.19.183.254:35699 was elected leader, sending a full tablet report...
I20260809 12:17:56.109607 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 1309
W20260809 12:17:56.131708 1196 connection.cc:535] server connection from 127.19.183.193:48283 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 12:17:56.132124 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 1505
I20260809 12:17:56.140409 1886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:56.133184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52002 (local address 127.19.183.194:44317)
0809 12:17:56.133783 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:56.133790 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:56.133820 (+ 30us) server_negotiation.cc:407] Connection header received
0809 12:17:56.133888 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:56.133895 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:56.134120 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:56.134257 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:56.134664 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:56.135748 (+ 1084us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:56.137004 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:56.137532 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:56.137649 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:56.138833 (+ 1184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:56.138857 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:56.138877 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:56.139560 (+ 683us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:56.139605 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:56.139626 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:56.139712 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:56.139899 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 12:17:56.140075 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":429,"thread_start_us":315,"threads_started":1}
I20260809 12:17:56.141712 1681 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:56.224864 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 1505
I20260809 12:17:56.250416 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 1689
I20260809 12:17:56.259117 1888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:56.251480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37206 (local address 127.19.183.195:42689)
0809 12:17:56.252194 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:56.252202 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:56.252234 (+ 32us) server_negotiation.cc:407] Connection header received
0809 12:17:56.252307 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:56.252313 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:56.252535 (+ 222us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:56.252762 (+ 227us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:56.253286 (+ 524us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:56.254341 (+ 1055us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:56.255697 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:17:56.256247 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:17:56.256382 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:17:56.257564 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:56.257573 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:56.257586 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:56.258196 (+ 610us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:56.258248 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:56.258268 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:56.258398 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:56.258559 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 12:17:56.258735 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":554,"thread_start_us":367,"threads_started":1}
I20260809 12:17:56.260381 1862 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:56.343523 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 1689
I20260809 12:17:56.365599 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 1163
I20260809 12:17:56.366983 1260 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:56.470533 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 1163
2026-08-09T12:17:56Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (2579 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T12:17:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:17:56Z Disabled control of system clock
I20260809 12:17:56.559834 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:36217
--webserver_interface=127.19.183.254
--webserver_port=0
--builtin_ntp_servers=127.19.183.212:40889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:36217
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 12:17:56.897138 1897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:56.897464 1897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:56.897615 1897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:56.904892 1897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:17:56.904994 1897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:56.905089 1897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:56.905194 1897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:17:56.905270 1897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:17:56.914384 1897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:36217
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:36217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:56.915791 1897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:56.917644 1897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:56.929797 1910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:56.929893 1905 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:56.929817 1908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:56.930583 1897 server_base.cc:1034] running on GCE node
I20260809 12:17:56.931366 1897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:56.932734 1897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:56.933940 1897 hybrid_clock.cc:648] HybridClock initialized: now 1786277876933894 us; error 42 us; skew 500 ppm
I20260809 12:17:56.934293 1897 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:17:56.940251 1897 webserver.cc:460] Webserver started at http://127.19.183.254:46579/ using document root <none> and password file <none>
I20260809 12:17:56.940995 1897 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:56.941120 1897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:56.941483 1897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:56.946651 1897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "d63476402e514864be951e3f6b09d52e"
format_stamp: "Formatted at 2026-08-09 12:17:56 on dist-test-slave-39l8"
server_key: "eda5484141fd81d2cc61b2629f1af7e4"
server_key_iv: "02513581e9a97443f82dbf01b5adb8e0"
server_key_version: "encryptionkey@0"
I20260809 12:17:56.947568 1897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "d63476402e514864be951e3f6b09d52e"
format_stamp: "Formatted at 2026-08-09 12:17:56 on dist-test-slave-39l8"
server_key: "eda5484141fd81d2cc61b2629f1af7e4"
server_key_iv: "02513581e9a97443f82dbf01b5adb8e0"
server_key_version: "encryptionkey@0"
I20260809 12:17:56.952708 1897 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260809 12:17:56.956164 1921 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:56.957448 1897 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:17:56.957651 1897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "d63476402e514864be951e3f6b09d52e"
format_stamp: "Formatted at 2026-08-09 12:17:56 on dist-test-slave-39l8"
server_key: "eda5484141fd81d2cc61b2629f1af7e4"
server_key_iv: "02513581e9a97443f82dbf01b5adb8e0"
server_key_version: "encryptionkey@0"
I20260809 12:17:56.957873 1897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:56.964704 1897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:56.965461 1897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:56.965768 1897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:56.984215 1897 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:36217
I20260809 12:17:56.984247 1985 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:36217 every 8 connection(s)
I20260809 12:17:56.985574 1897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:17:56.989266 1987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:17:56.992411 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 1897
I20260809 12:17:56.992633 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:17:56.993467 20191 external_mini_cluster.cc:1428] Setting key c78f626b6bd7abf8e64b9848b530ddce
I20260809 12:17:57.001802 1990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:56.995624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49544 (local address 127.19.183.254:36217)
0809 12:17:56.996681 (+ 1057us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:56.996700 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:56.996741 (+ 41us) server_negotiation.cc:407] Connection header received
0809 12:17:56.998328 (+ 1587us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:56.998350 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:56.998835 (+ 485us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:56.999154 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:57.000253 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:57.000259 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:57.000271 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:57.000286 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:57.000344 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:57.000359 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:57.000534 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:57.000665 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 12:17:57.000947 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":591,"thread_start_us":374,"threads_started":1}
I20260809 12:17:57.002360 1987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d63476402e514864be951e3f6b09d52e: Bootstrap starting.
I20260809 12:17:57.005412 1987 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d63476402e514864be951e3f6b09d52e: Neither blocks nor log segments found. Creating new log.
I20260809 12:17:57.006405 1987 log.cc:824] T 00000000000000000000000000000000 P d63476402e514864be951e3f6b09d52e: Log is configured to *not* fsync() on all Append() calls
I20260809 12:17:57.008994 1987 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d63476402e514864be951e3f6b09d52e: No bootstrap required, opened a new log
I20260809 12:17:57.016098 1987 raft_consensus.cc:348] T 00000000000000000000000000000000 P d63476402e514864be951e3f6b09d52e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63476402e514864be951e3f6b09d52e" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36217 } }
I20260809 12:17:57.016450 1987 raft_consensus.cc:374] T 00000000000000000000000000000000 P d63476402e514864be951e3f6b09d52e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:17:57.016573 1987 raft_consensus.cc:729] T 00000000000000000000000000000000 P d63476402e514864be951e3f6b09d52e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d63476402e514864be951e3f6b09d52e, State: Initialized, Role: FOLLOWER
I20260809 12:17:57.017102 1987 consensus_queue.cc:260] T 00000000000000000000000000000000 P d63476402e514864be951e3f6b09d52e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63476402e514864be951e3f6b09d52e" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36217 } }
I20260809 12:17:57.017342 1987 raft_consensus.cc:388] T 00000000000000000000000000000000 P d63476402e514864be951e3f6b09d52e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:17:57.017482 1987 raft_consensus.cc:482] T 00000000000000000000000000000000 P d63476402e514864be951e3f6b09d52e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:17:57.017660 1987 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d63476402e514864be951e3f6b09d52e [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:17:57.019892 1987 raft_consensus.cc:504] T 00000000000000000000000000000000 P d63476402e514864be951e3f6b09d52e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63476402e514864be951e3f6b09d52e" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36217 } }
I20260809 12:17:57.020385 1987 leader_election.cc:302] T 00000000000000000000000000000000 P d63476402e514864be951e3f6b09d52e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d63476402e514864be951e3f6b09d52e; no voters:
I20260809 12:17:57.020969 1987 leader_election.cc:288] T 00000000000000000000000000000000 P d63476402e514864be951e3f6b09d52e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:17:57.021075 1994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d63476402e514864be951e3f6b09d52e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:17:57.021826 1994 raft_consensus.cc:686] T 00000000000000000000000000000000 P d63476402e514864be951e3f6b09d52e [term 1 LEADER]: Becoming Leader. State: Replica: d63476402e514864be951e3f6b09d52e, State: Running, Role: LEADER
I20260809 12:17:57.022382 1994 consensus_queue.cc:237] T 00000000000000000000000000000000 P d63476402e514864be951e3f6b09d52e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63476402e514864be951e3f6b09d52e" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36217 } }
I20260809 12:17:57.023116 1987 sys_catalog.cc:564] T 00000000000000000000000000000000 P d63476402e514864be951e3f6b09d52e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:17:57.025198 1997 sys_catalog.cc:455] T 00000000000000000000000000000000 P d63476402e514864be951e3f6b09d52e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d63476402e514864be951e3f6b09d52e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63476402e514864be951e3f6b09d52e" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36217 } } }
I20260809 12:17:57.025657 1997 sys_catalog.cc:458] T 00000000000000000000000000000000 P d63476402e514864be951e3f6b09d52e [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:57.025797 1999 sys_catalog.cc:455] T 00000000000000000000000000000000 P d63476402e514864be951e3f6b09d52e [sys.catalog]: SysCatalogTable state changed. Reason: New leader d63476402e514864be951e3f6b09d52e. Latest consensus state: current_term: 1 leader_uuid: "d63476402e514864be951e3f6b09d52e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d63476402e514864be951e3f6b09d52e" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36217 } } }
I20260809 12:17:57.026160 1999 sys_catalog.cc:458] T 00000000000000000000000000000000 P d63476402e514864be951e3f6b09d52e [sys.catalog]: This master's current role is: LEADER
I20260809 12:17:57.027748 2018 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:17:57.032922 2018 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:17:57.041924 2018 catalog_manager.cc:1347] Generated new cluster ID: 64b2301b6a3742f1978d573676798adb
I20260809 12:17:57.042054 2018 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:17:57.050144 2018 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:17:57.050536 2018 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:17:57.058243 2018 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d63476402e514864be951e3f6b09d52e: Generated new TSK 0
I20260809 12:17:57.059182 2018 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:17:57.068408 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:36217
--builtin_ntp_servers=127.19.183.212:40889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 12:17:57.425314 2034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:57.425596 2034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:57.425688 2034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:57.434296 2034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:57.434419 2034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:57.434556 2034 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:17:57.444304 2034 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:36217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:57.445555 2034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:57.447381 2034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:57.461385 2043 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:57.461495 2045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:57.461542 2048 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:57.462307 2034 server_base.cc:1034] running on GCE node
I20260809 12:17:57.463011 2034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:57.464059 2034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:57.465298 2034 hybrid_clock.cc:648] HybridClock initialized: now 1786277877465242 us; error 51 us; skew 500 ppm
I20260809 12:17:57.465629 2034 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:17:57.467949 2034 webserver.cc:460] Webserver started at http://127.19.183.193:34815/ using document root <none> and password file <none>
I20260809 12:17:57.468693 2034 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:57.468871 2034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:57.469210 2034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:57.472173 2034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "39f8a0d68822407aa55f10286dea578b"
format_stamp: "Formatted at 2026-08-09 12:17:57 on dist-test-slave-39l8"
server_key: "31a5622c3211e2c5036efc62295af7c5"
server_key_iv: "21b1d22f514c2cd9b0cc425dc6c60aaa"
server_key_version: "encryptionkey@0"
I20260809 12:17:57.473095 2034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "39f8a0d68822407aa55f10286dea578b"
format_stamp: "Formatted at 2026-08-09 12:17:57 on dist-test-slave-39l8"
server_key: "31a5622c3211e2c5036efc62295af7c5"
server_key_iv: "21b1d22f514c2cd9b0cc425dc6c60aaa"
server_key_version: "encryptionkey@0"
I20260809 12:17:57.478369 2034 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 12:17:57.481947 2058 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:57.483381 2034 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:17:57.483558 2034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "39f8a0d68822407aa55f10286dea578b"
format_stamp: "Formatted at 2026-08-09 12:17:57 on dist-test-slave-39l8"
server_key: "31a5622c3211e2c5036efc62295af7c5"
server_key_iv: "21b1d22f514c2cd9b0cc425dc6c60aaa"
server_key_version: "encryptionkey@0"
I20260809 12:17:57.483798 2034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:57.489421 2034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:57.490221 2034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:57.490506 2034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:57.491569 2034 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:57.493016 2034 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:57.493106 2034 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:57.493201 2034 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:57.493294 2034 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:57.530580 2034 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:37151
I20260809 12:17:57.530637 2233 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:37151 every 8 connection(s)
I20260809 12:17:57.531972 2034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:17:57.534368 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 2034
I20260809 12:17:57.534575 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:17:57.535451 20191 external_mini_cluster.cc:1428] Setting key 1b8f4806183bc8ef2944d6480370ddef
I20260809 12:17:57.538655 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:36217
--builtin_ntp_servers=127.19.183.212:40889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 12:17:57.541091 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:57.534606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:47371 (local address 127.19.183.254:36217)
0809 12:17:57.535171 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:57.535178 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:57.536508 (+ 1330us) server_negotiation.cc:407] Connection header received
0809 12:17:57.538525 (+ 2017us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:57.538532 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:57.538601 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:57.538750 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:57.540220 (+ 1470us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:57.540230 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:57.540238 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:57.540262 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:57.540320 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:57.540337 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:57.540478 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:57.540651 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 12:17:57.540723 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":408,"thread_start_us":279,"threads_started":1}
I20260809 12:17:57.541908 2242 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:57.535265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:36217 (local address 127.19.183.193:47371)
0809 12:17:57.536344 (+ 1079us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:57.536392 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:57.538246 (+ 1854us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:57.539070 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:57.539079 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:57.539174 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:57.539889 (+ 715us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:17:57.539891 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:57.539929 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:17:57.540019 (+ 90us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:57.540450 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:57.540458 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:57.540464 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 12:17:57.540688 (+ 224us) client_negotiation.cc:240] Negotiation successful
0809 12:17:57.540884 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":355,"threads_started":1}
I20260809 12:17:57.543480 2235 heartbeater.cc:347] Connected to a master server at 127.19.183.254:36217
I20260809 12:17:57.543797 2235 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:57.544394 2235 heartbeater.cc:511] Master 127.19.183.254:36217 requested a full tablet report, sending...
I20260809 12:17:57.546420 1944 ts_manager.cc:194] Registered new tserver with Master: 39f8a0d68822407aa55f10286dea578b (127.19.183.193:37151)
W20260809 12:17:57.880682 2244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:57.880947 2244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:57.881049 2244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:57.890321 2244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:57.890470 2244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:57.890676 2244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:17:57.902628 2244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:36217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:57.903949 2244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:57.905664 2244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:57.919314 2256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:57.919392 2251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:57.919286 2252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:57.920161 2244 server_base.cc:1034] running on GCE node
I20260809 12:17:57.920789 2244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:57.921779 2244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:57.923015 2244 hybrid_clock.cc:648] HybridClock initialized: now 1786277877922966 us; error 62 us; skew 500 ppm
I20260809 12:17:57.923379 2244 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:17:57.925870 2244 webserver.cc:460] Webserver started at http://127.19.183.194:41899/ using document root <none> and password file <none>
I20260809 12:17:57.926654 2244 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:57.926819 2244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:57.927211 2244 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:57.930337 2244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "19e6abeac3a240bfb603213d2f9772fa"
format_stamp: "Formatted at 2026-08-09 12:17:57 on dist-test-slave-39l8"
server_key: "06920143f0c1661905e9cbe499db1e09"
server_key_iv: "9c44bfc063e27a20ef7608e148282f86"
server_key_version: "encryptionkey@0"
I20260809 12:17:57.931201 2244 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "19e6abeac3a240bfb603213d2f9772fa"
format_stamp: "Formatted at 2026-08-09 12:17:57 on dist-test-slave-39l8"
server_key: "06920143f0c1661905e9cbe499db1e09"
server_key_iv: "9c44bfc063e27a20ef7608e148282f86"
server_key_version: "encryptionkey@0"
I20260809 12:17:57.936832 2244 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:17:57.940286 2266 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:57.941671 2244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:17:57.941856 2244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "19e6abeac3a240bfb603213d2f9772fa"
format_stamp: "Formatted at 2026-08-09 12:17:57 on dist-test-slave-39l8"
server_key: "06920143f0c1661905e9cbe499db1e09"
server_key_iv: "9c44bfc063e27a20ef7608e148282f86"
server_key_version: "encryptionkey@0"
I20260809 12:17:57.942081 2244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:57.947818 2244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:57.948625 2244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:57.948910 2244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:57.949797 2244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:57.951273 2244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:57.951395 2244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:57.951534 2244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:57.951632 2244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:57.991235 2244 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:41115
I20260809 12:17:57.991323 2444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:41115 every 8 connection(s)
I20260809 12:17:57.992579 2244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:17:58.000021 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 2244
I20260809 12:17:58.000202 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:17:58.000833 20191 external_mini_cluster.cc:1428] Setting key 2cb82b69daeb4c332fc3e1ceb3f13423
I20260809 12:17:58.002152 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:57.995372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:45075 (local address 127.19.183.254:36217)
0809 12:17:57.995584 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:57.995591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:57.997054 (+ 1463us) server_negotiation.cc:407] Connection header received
0809 12:17:57.999531 (+ 2477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:57.999536 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:57.999649 (+ 113us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:57.999781 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:58.001297 (+ 1516us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:58.001302 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:58.001307 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:58.001326 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:58.001365 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:58.001382 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:58.001507 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:58.001762 (+ 255us) server_negotiation.cc:299] Negotiation successful
0809 12:17:58.001828 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:17:58.003257 2450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:57.995861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:36217 (local address 127.19.183.194:45075)
0809 12:17:57.996886 (+ 1025us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:57.996929 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:57.999227 (+ 2298us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:58.000032 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:58.000043 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:58.000167 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:58.000927 (+ 760us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:17:58.000931 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:58.000982 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:17:58.001089 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:58.001483 (+ 394us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:58.001495 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:58.001503 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 12:17:58.001829 (+ 326us) client_negotiation.cc:240] Negotiation successful
0809 12:17:58.002091 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":301,"threads_started":1}
I20260809 12:17:58.004045 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:36217
--builtin_ntp_servers=127.19.183.212:40889
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 12:17:58.004776 2446 heartbeater.cc:347] Connected to a master server at 127.19.183.254:36217
I20260809 12:17:58.005092 2446 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:58.005844 2446 heartbeater.cc:511] Master 127.19.183.254:36217 requested a full tablet report, sending...
I20260809 12:17:58.007339 1944 ts_manager.cc:194] Registered new tserver with Master: 19e6abeac3a240bfb603213d2f9772fa (127.19.183.194:41115)
W20260809 12:17:58.355626 2451 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:17:58.355962 2451 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:17:58.356137 2451 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:17:58.363476 2451 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:17:58.363612 2451 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:17:58.363782 2451 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:17:58.372752 2451 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40889
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:36217
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:17:58.374047 2451 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:17:58.375871 2451 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:17:58.388897 2463 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:58.388937 2465 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:17:58.388850 2467 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:17:58.389899 2451 server_base.cc:1034] running on GCE node
I20260809 12:17:58.390453 2451 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:17:58.391389 2451 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:17:58.392607 2451 hybrid_clock.cc:648] HybridClock initialized: now 1786277878392558 us; error 40 us; skew 500 ppm
I20260809 12:17:58.392939 2451 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:17:58.395226 2451 webserver.cc:460] Webserver started at http://127.19.183.195:35175/ using document root <none> and password file <none>
I20260809 12:17:58.395967 2451 fs_manager.cc:359] Metadata directory not provided
I20260809 12:17:58.396092 2451 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:17:58.396432 2451 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:17:58.399501 2451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "473ab0be4b2746769edb87913c7e9889"
format_stamp: "Formatted at 2026-08-09 12:17:58 on dist-test-slave-39l8"
server_key: "0dcf62577edac606d29531aa2ef8d692"
server_key_iv: "1c0cf3ffcf1870d45a12ef5bc2c5436b"
server_key_version: "encryptionkey@0"
I20260809 12:17:58.400316 2451 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "473ab0be4b2746769edb87913c7e9889"
format_stamp: "Formatted at 2026-08-09 12:17:58 on dist-test-slave-39l8"
server_key: "0dcf62577edac606d29531aa2ef8d692"
server_key_iv: "1c0cf3ffcf1870d45a12ef5bc2c5436b"
server_key_version: "encryptionkey@0"
I20260809 12:17:58.405587 2451 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 12:17:58.409035 2482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:58.410377 2451 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:17:58.410570 2451 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "473ab0be4b2746769edb87913c7e9889"
format_stamp: "Formatted at 2026-08-09 12:17:58 on dist-test-slave-39l8"
server_key: "0dcf62577edac606d29531aa2ef8d692"
server_key_iv: "1c0cf3ffcf1870d45a12ef5bc2c5436b"
server_key_version: "encryptionkey@0"
I20260809 12:17:58.410894 2451 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:17:58.416661 2451 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:17:58.417368 2451 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:17:58.417584 2451 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:17:58.418467 2451 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:17:58.419822 2451 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:17:58.419910 2451 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:58.419989 2451 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:17:58.420044 2451 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:17:58.453239 2451 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:40433
I20260809 12:17:58.453302 2656 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:40433 every 8 connection(s)
I20260809 12:17:58.454607 2451 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:17:58.456670 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 2451
I20260809 12:17:58.456851 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:17:58.457484 20191 external_mini_cluster.cc:1428] Setting key 27e5487d54f0ec2cf8bf1b8004d2fcb8
I20260809 12:17:58.463418 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:58.457540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:40911 (local address 127.19.183.254:36217)
0809 12:17:58.457767 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:58.457773 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:58.459441 (+ 1668us) server_negotiation.cc:407] Connection header received
0809 12:17:58.461290 (+ 1849us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:58.461294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:58.461356 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:58.461439 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:58.462755 (+ 1316us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:58.462759 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:58.462763 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:58.462778 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:58.462809 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:58.462821 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:58.462912 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:58.463112 (+ 200us) server_negotiation.cc:299] Negotiation successful
0809 12:17:58.463161 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 12:17:58.464336 2659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:58.458059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:36217 (local address 127.19.183.195:40911)
0809 12:17:58.459263 (+ 1204us) negotiation.cc:107] Waiting for socket to connect
0809 12:17:58.459307 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:17:58.461048 (+ 1741us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:17:58.461647 (+ 599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:17:58.461654 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:17:58.461736 (+ 82us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:17:58.462381 (+ 645us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:17:58.462384 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:17:58.462420 (+ 36us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:17:58.462554 (+ 134us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:17:58.462914 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:17:58.462922 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:17:58.462928 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 12:17:58.463205 (+ 277us) client_negotiation.cc:240] Negotiation successful
0809 12:17:58.463393 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"thread_start_us":384,"threads_started":1}
I20260809 12:17:58.465659 2657 heartbeater.cc:347] Connected to a master server at 127.19.183.254:36217
I20260809 12:17:58.465950 2657 heartbeater.cc:464] Registering TS with master...
I20260809 12:17:58.466499 2657 heartbeater.cc:511] Master 127.19.183.254:36217 requested a full tablet report, sending...
I20260809 12:17:58.467828 1944 ts_manager.cc:194] Registered new tserver with Master: 473ab0be4b2746769edb87913c7e9889 (127.19.183.195:40433)
I20260809 12:17:58.472633 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 12:17:58.478449 2241 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:17:58.477064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49550 (local address 127.19.183.254:36217)
0809 12:17:58.477242 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:58.477247 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:58.477722 (+ 475us) server_negotiation.cc:407] Connection header received
0809 12:17:58.477848 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:58.477851 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:58.477900 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:58.477976 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:58.478270 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49550: BlockingRecv error: recv got EOF from 127.0.0.1:49550 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 12:17:58.479324 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 2034
I20260809 12:17:58.482616 2670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:58.480261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50828 (local address 127.19.183.193:37151)
0809 12:17:58.480798 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:58.480805 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:58.480829 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:17:58.480886 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:58.480891 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:58.481066 (+ 175us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:58.481197 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:58.482039 (+ 842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:58.482044 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:58.482054 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:58.482072 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:58.482118 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:58.482133 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:58.482236 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:58.482319 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 12:17:58.482395 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":275,"threads_started":1}
I20260809 12:17:58.483834 2229 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:58.564324 2235 heartbeater.cc:503] Master 127.19.183.254:36217 was elected leader, sending a full tablet report...
I20260809 12:17:58.565769 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 2034
W20260809 12:17:58.590458 1924 connection.cc:535] server connection from 127.19.183.193:47371 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 12:17:58.590771 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 2244
I20260809 12:17:58.594488 2673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:58.591869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55038 (local address 127.19.183.194:41115)
0809 12:17:58.592469 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:58.592476 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:58.592518 (+ 42us) server_negotiation.cc:407] Connection header received
0809 12:17:58.592585 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:58.592591 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:58.592810 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:58.592940 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:58.593842 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:58.593849 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:58.593861 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:58.593882 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:58.593928 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:58.593943 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:58.594055 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:58.594154 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 12:17:58.594236 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":322,"threads_started":1}
I20260809 12:17:58.595945 2439 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:58.679014 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 2244
I20260809 12:17:58.704475 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 2451
I20260809 12:17:58.708801 2677 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:17:58.705591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57768 (local address 127.19.183.195:40433)
0809 12:17:58.706289 (+ 698us) server_negotiation.cc:206] Beginning negotiation
0809 12:17:58.706296 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:17:58.706326 (+ 30us) server_negotiation.cc:407] Connection header received
0809 12:17:58.706401 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:17:58.706407 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:17:58.706630 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:17:58.706832 (+ 202us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:17:58.708034 (+ 1202us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:17:58.708040 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:17:58.708051 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:17:58.708068 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:17:58.708117 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:17:58.708131 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:17:58.708319 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0809 12:17:58.708395 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 12:17:58.708501 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":557,"thread_start_us":341,"threads_started":1}
I20260809 12:17:58.710019 2650 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:58.791935 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 2451
I20260809 12:17:58.815841 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 1897
I20260809 12:17:58.817083 1980 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:17:58.920390 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 1897
2026-08-09T12:17:58Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (2454 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:17:59 dist-test-slave-39l8 krb5kdc[2683](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:17:59 dist-test-slave-39l8 krb5kdc[2683](info): set up 2 sockets
Aug 09 12:17:59 dist-test-slave-39l8 krb5kdc[2683](info): commencing operation
krb5kdc: starting...
W20260809 12:18:00.234692 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.261s user 0.003s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:18:00 dist-test-slave-39l8 krb5kdc[2683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:18:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:18:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:18:00.531471 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:40669
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:33821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:40669
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:18:00.874749 2705 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:18:00.875074 2705 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:18:00.875236 2705 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:18:00.882282 2705 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:18:00.882422 2705 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:18:00.882547 2705 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:18:00.882642 2705 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:18:00.882740 2705 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:18:00.892333 2705 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:33821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:40669
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:40669
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:18:00.893687 2705 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:18:00.895030 2705 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:18:00.906279 2714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:18:00.906373 2715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:18:00.906327 2718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:18:00.907585 2705 server_base.cc:1034] running on GCE node
I20260809 12:18:00.908313 2705 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:18:00.909667 2705 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:18:00.910912 2705 hybrid_clock.cc:648] HybridClock initialized: now 1786277880910875 us; error 51 us; skew 500 ppm
I20260809 12:18:00.911270 2705 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:18:00 dist-test-slave-39l8 krb5kdc[2683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277880, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:18:00.914361 2705 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:18:00.915966 2705 webserver.cc:460] Webserver started at http://127.19.183.254:38067/ using document root <none> and password file <none>
I20260809 12:18:00.916687 2705 fs_manager.cc:359] Metadata directory not provided
I20260809 12:18:00.916838 2705 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:18:00.917172 2705 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:18:00.920307 2705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "8119f296c59d40ee833e7ebfb25cdfcd"
format_stamp: "Formatted at 2026-08-09 12:18:00 on dist-test-slave-39l8"
server_key: "d081fe8f5a4922c8ab7eff0aaf70804d"
server_key_iv: "b4f646b6bb9ec3224c36b463f236e698"
server_key_version: "encryptionkey@0"
I20260809 12:18:00.921092 2705 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "8119f296c59d40ee833e7ebfb25cdfcd"
format_stamp: "Formatted at 2026-08-09 12:18:00 on dist-test-slave-39l8"
server_key: "d081fe8f5a4922c8ab7eff0aaf70804d"
server_key_iv: "b4f646b6bb9ec3224c36b463f236e698"
server_key_version: "encryptionkey@0"
I20260809 12:18:00.926258 2705 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:18:00.929858 2733 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:18:00.931246 2705 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:18:00.931424 2705 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "8119f296c59d40ee833e7ebfb25cdfcd"
format_stamp: "Formatted at 2026-08-09 12:18:00 on dist-test-slave-39l8"
server_key: "d081fe8f5a4922c8ab7eff0aaf70804d"
server_key_iv: "b4f646b6bb9ec3224c36b463f236e698"
server_key_version: "encryptionkey@0"
I20260809 12:18:00.931623 2705 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:18:00.945433 2705 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:18:00.949261 2705 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:18:00.949533 2705 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:18:00.968019 2705 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:40669
I20260809 12:18:00.968050 2797 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:40669 every 8 connection(s)
I20260809 12:18:00.969427 2705 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:18:00.970013 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 2705
I20260809 12:18:00.970228 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:18:00.971083 20191 external_mini_cluster.cc:1428] Setting key faabd4a5706308e28154d520855aaa67
I20260809 12:18:00.973964 2799 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:18:00.985453 2799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd: Bootstrap starting.
Aug 09 12:18:00 dist-test-slave-39l8 krb5kdc[2683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:18:00.989104 2799 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd: Neither blocks nor log segments found. Creating new log.
I20260809 12:18:00.990531 2799 log.cc:824] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd: Log is configured to *not* fsync() on all Append() calls
I20260809 12:18:00.993433 2799 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd: No bootstrap required, opened a new log
I20260809 12:18:00.995460 2805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:00.973163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35646 (local address 127.19.183.254:40669)
0809 12:18:00.974023 (+ 860us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:00.974041 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:00.974096 (+ 55us) server_negotiation.cc:407] Connection header received
0809 12:18:00.976796 (+ 2700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:00.976831 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:00.977351 (+ 520us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:00.977776 (+ 425us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:18:00.979348 (+ 1572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:00.980674 (+ 1326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:00.982063 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:00.982872 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:00.983090 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:00.987562 (+ 4472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:18:00.987570 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:18:00.987583 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:18:00.987629 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:18:00.991433 (+ 3804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:00.992138 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:00.992147 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:00.992156 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:00.992325 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:00.992731 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:00.992737 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:00.992740 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:00.993241 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:18:00.993573 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:00.994021 (+ 448us) server_negotiation.cc:299] Negotiation successful
0809 12:18:00.994434 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":375,"threads_started":1}
I20260809 12:18:01.001255 2799 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8119f296c59d40ee833e7ebfb25cdfcd" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40669 } }
I20260809 12:18:01.001629 2799 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:18:01.001744 2799 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8119f296c59d40ee833e7ebfb25cdfcd, State: Initialized, Role: FOLLOWER
I20260809 12:18:01.002282 2799 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8119f296c59d40ee833e7ebfb25cdfcd" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40669 } }
I20260809 12:18:01.002575 2799 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:18:01.002735 2799 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:18:01.002928 2799 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:18:01.005512 2799 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8119f296c59d40ee833e7ebfb25cdfcd" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40669 } }
I20260809 12:18:01.005986 2799 leader_election.cc:302] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8119f296c59d40ee833e7ebfb25cdfcd; no voters:
I20260809 12:18:01.006563 2799 leader_election.cc:288] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:18:01.006695 2807 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:18:01.007447 2807 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd [term 1 LEADER]: Becoming Leader. State: Replica: 8119f296c59d40ee833e7ebfb25cdfcd, State: Running, Role: LEADER
I20260809 12:18:01.008008 2807 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8119f296c59d40ee833e7ebfb25cdfcd" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40669 } }
I20260809 12:18:01.008658 2799 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:18:01.010922 2808 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8119f296c59d40ee833e7ebfb25cdfcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8119f296c59d40ee833e7ebfb25cdfcd" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40669 } } }
I20260809 12:18:01.011592 2808 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd [sys.catalog]: This master's current role is: LEADER
I20260809 12:18:01.011740 2809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8119f296c59d40ee833e7ebfb25cdfcd. Latest consensus state: current_term: 1 leader_uuid: "8119f296c59d40ee833e7ebfb25cdfcd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8119f296c59d40ee833e7ebfb25cdfcd" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40669 } } }
I20260809 12:18:01.012483 2809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd [sys.catalog]: This master's current role is: LEADER
I20260809 12:18:01.017027 2818 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:18:01.022544 2818 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:18:01.031677 2818 catalog_manager.cc:1347] Generated new cluster ID: 7983b3877e6942c494a998f75e708453
I20260809 12:18:01.031795 2818 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:18:01.056385 2818 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:18:01.057644 2818 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:18:01.064900 2818 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd: Generated new TSK 0
I20260809 12:18:01.065657 2818 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
I20260809 12:18:01.140657 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40669
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:33821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:18:01.485301 2833 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:18:01.485570 2833 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:18:01.485733 2833 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:18:01.492980 2833 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:18:01.493114 2833 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:18:01.493310 2833 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:18:01.502790 2833 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:33821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:18:01.504024 2833 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:18:01.505374 2833 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:18:01.519001 2842 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:18:01.519204 2846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:18:01.519130 2844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:18:01.520066 2833 server_base.cc:1034] running on GCE node
I20260809 12:18:01.520638 2833 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:18:01.521535 2833 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:18:01.522804 2833 hybrid_clock.cc:648] HybridClock initialized: now 1786277881522745 us; error 64 us; skew 500 ppm
I20260809 12:18:01.523135 2833 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:18:01 dist-test-slave-39l8 krb5kdc[2683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277881, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:18:01.526401 2833 init.cc:377] Logged in from keytab as kudu/127.19.183.193@KRBTEST.COM (short username kudu)
I20260809 12:18:01.527973 2833 webserver.cc:460] Webserver started at http://127.19.183.193:38547/ using document root <none> and password file <none>
I20260809 12:18:01.528708 2833 fs_manager.cc:359] Metadata directory not provided
I20260809 12:18:01.528867 2833 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:18:01.529194 2833 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:18:01.532281 2833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "ed3c10a3f08d4b5bbd6fdf04e9443ce0"
format_stamp: "Formatted at 2026-08-09 12:18:01 on dist-test-slave-39l8"
server_key: "b80e60d7f77c48728d02fff2a840fab8"
server_key_iv: "7004a6e5f33a059d23c14f0fde9936df"
server_key_version: "encryptionkey@0"
I20260809 12:18:01.533103 2833 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "ed3c10a3f08d4b5bbd6fdf04e9443ce0"
format_stamp: "Formatted at 2026-08-09 12:18:01 on dist-test-slave-39l8"
server_key: "b80e60d7f77c48728d02fff2a840fab8"
server_key_iv: "7004a6e5f33a059d23c14f0fde9936df"
server_key_version: "encryptionkey@0"
I20260809 12:18:01.539014 2833 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.004s sys 0.004s
I20260809 12:18:01.542462 2853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:18:01.543808 2833 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:18:01.543987 2833 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "ed3c10a3f08d4b5bbd6fdf04e9443ce0"
format_stamp: "Formatted at 2026-08-09 12:18:01 on dist-test-slave-39l8"
server_key: "b80e60d7f77c48728d02fff2a840fab8"
server_key_iv: "7004a6e5f33a059d23c14f0fde9936df"
server_key_version: "encryptionkey@0"
I20260809 12:18:01.544229 2833 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:18:01.573709 2833 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:18:01.577687 2833 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:18:01.578027 2833 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:18:01.579071 2833 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:18:01.580433 2833 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:18:01.580540 2833 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:18:01.580720 2833 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:18:01.580796 2833 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:18:01.615301 2833 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:39313
I20260809 12:18:01.615356 3013 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:39313 every 8 connection(s)
I20260809 12:18:01.616763 2833 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:18:01.619443 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 2833
I20260809 12:18:01.619709 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:18:01.620465 20191 external_mini_cluster.cc:1428] Setting key 92244afddd566258a728d5d8826ad092
Aug 09 12:18:01 dist-test-slave-39l8 krb5kdc[2683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277881, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:18:01.639919 3020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:01.619571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:43567 (local address 127.19.183.254:40669)
0809 12:18:01.620083 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:01.620090 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:01.621310 (+ 1220us) server_negotiation.cc:407] Connection header received
0809 12:18:01.623704 (+ 2394us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:01.623708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:01.623771 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:01.623855 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:18:01.626027 (+ 2172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:01.626751 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:01.629236 (+ 2485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:01.629613 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:01.629714 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:01.633926 (+ 4212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:18:01.633931 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:18:01.633934 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:18:01.633967 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:18:01.636729 (+ 2762us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:01.637521 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:01.637527 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:01.637532 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:01.637661 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:01.638094 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:01.638099 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:01.638103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:01.638373 (+ 270us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:18:01.638538 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:01.639304 (+ 766us) server_negotiation.cc:299] Negotiation successful
0809 12:18:01.639526 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":243,"threads_started":1}
I20260809 12:18:01.640852 3021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:01.620229 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:40669 (local address 127.19.183.193:43567)
0809 12:18:01.621130 (+ 901us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:01.621196 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 12:18:01.623378 (+ 2182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:18:01.624140 (+ 762us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:18:01.624155 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:18:01.624953 (+ 798us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:18:01.625776 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:01.625802 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:01.626934 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:01.626941 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:01.629008 (+ 2067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:01.629026 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:01.629849 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:01.629858 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:01.630184 (+ 326us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:01.631004 (+ 820us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:18:01.631039 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:18:01.633709 (+ 2670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:18:01.636928 (+ 3219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:18:01.636936 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:18:01.636951 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:18:01.637388 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:18:01.637777 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:18:01.637782 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:18:01.637786 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:18:01.637974 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:18:01.638513 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:18:01.638521 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:18:01.638927 (+ 406us) client_negotiation.cc:769] Sending connection context
0809 12:18:01.639207 (+ 280us) client_negotiation.cc:240] Negotiation successful
0809 12:18:01.639736 (+ 529us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":334,"threads_started":1}
I20260809 12:18:01.642510 3016 heartbeater.cc:347] Connected to a master server at 127.19.183.254:40669
I20260809 12:18:01.642853 3016 heartbeater.cc:464] Registering TS with master...
I20260809 12:18:01.643564 3016 heartbeater.cc:511] Master 127.19.183.254:40669 requested a full tablet report, sending...
I20260809 12:18:01.645936 2752 ts_manager.cc:194] Registered new tserver with Master: ed3c10a3f08d4b5bbd6fdf04e9443ce0 (127.19.183.193:39313)
I20260809 12:18:01.648350 2752 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:43567
WARNING: no policy specified for kudu/127.19.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
I20260809 12:18:01.687533 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40669
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:33821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:18:02.031523 3025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:18:02.031833 3025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:18:02.032013 3025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:18:02.039603 3025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:18:02.039767 3025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:18:02.039943 3025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:18:02.050004 3025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:33821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:18:02.051375 3025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:18:02.052731 3025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:18:02.066162 3033 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:18:02.066202 3035 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:18:02.066238 3037 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:18:02.067420 3025 server_base.cc:1034] running on GCE node
I20260809 12:18:02.068011 3025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:18:02.068977 3025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:18:02.070197 3025 hybrid_clock.cc:648] HybridClock initialized: now 1786277882070144 us; error 54 us; skew 500 ppm
I20260809 12:18:02.070508 3025 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:18:02 dist-test-slave-39l8 krb5kdc[2683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277882, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:18:02.076582 3025 init.cc:377] Logged in from keytab as kudu/127.19.183.194@KRBTEST.COM (short username kudu)
I20260809 12:18:02.078002 3025 webserver.cc:460] Webserver started at http://127.19.183.194:40451/ using document root <none> and password file <none>
I20260809 12:18:02.078650 3025 fs_manager.cc:359] Metadata directory not provided
I20260809 12:18:02.078819 3025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:18:02.079092 3025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:18:02.081961 3025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "f6fafaec836a4b6ea56041f9757742ce"
format_stamp: "Formatted at 2026-08-09 12:18:02 on dist-test-slave-39l8"
server_key: "01452d4b15d3795a590c097721a2ce03"
server_key_iv: "10e95aa7b37b25421d4e89dd6f9baf8c"
server_key_version: "encryptionkey@0"
I20260809 12:18:02.082751 3025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "f6fafaec836a4b6ea56041f9757742ce"
format_stamp: "Formatted at 2026-08-09 12:18:02 on dist-test-slave-39l8"
server_key: "01452d4b15d3795a590c097721a2ce03"
server_key_iv: "10e95aa7b37b25421d4e89dd6f9baf8c"
server_key_version: "encryptionkey@0"
I20260809 12:18:02.087850 3025 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:18:02.091199 3048 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:18:02.092523 3025 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 12:18:02.092708 3025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "f6fafaec836a4b6ea56041f9757742ce"
format_stamp: "Formatted at 2026-08-09 12:18:02 on dist-test-slave-39l8"
server_key: "01452d4b15d3795a590c097721a2ce03"
server_key_iv: "10e95aa7b37b25421d4e89dd6f9baf8c"
server_key_version: "encryptionkey@0"
I20260809 12:18:02.092901 3025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:18:02.114904 3025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:18:02.119601 3025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:18:02.119915 3025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:18:02.120937 3025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:18:02.122331 3025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:18:02.122411 3025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:18:02.122560 3025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:18:02.122642 3025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:18:02.158083 3025 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:38501
I20260809 12:18:02.158133 3217 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:38501 every 8 connection(s)
I20260809 12:18:02.159428 3025 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:18:02.167162 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 3025
I20260809 12:18:02.167351 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:18:02.167807 20191 external_mini_cluster.cc:1428] Setting key 2b6f07613ff953707326235d0b88e429
Aug 09 12:18:02 dist-test-slave-39l8 krb5kdc[2683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277882, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:18:02.182590 3223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:02.162227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:57859 (local address 127.19.183.254:40669)
0809 12:18:02.162665 (+ 438us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:02.162670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:02.163708 (+ 1038us) server_negotiation.cc:407] Connection header received
0809 12:18:02.165579 (+ 1871us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:02.165584 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:02.165676 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:02.165798 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:18:02.167408 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:02.168242 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:02.170183 (+ 1941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:02.170591 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:02.170685 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:02.175135 (+ 4450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:18:02.175144 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:18:02.175152 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:18:02.175199 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:18:02.178911 (+ 3712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:02.179806 (+ 895us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:02.179814 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:02.179820 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:02.179985 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:02.180459 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:02.180466 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:02.180472 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:02.180891 (+ 419us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:18:02.181061 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:02.181856 (+ 795us) server_negotiation.cc:299] Negotiation successful
0809 12:18:02.182114 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":225,"threads_started":1}
I20260809 12:18:02.183512 3224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:02.162837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:40669 (local address 127.19.183.194:57859)
0809 12:18:02.163553 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:02.163590 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 12:18:02.165307 (+ 1717us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:18:02.166040 (+ 733us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:18:02.166048 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:18:02.166542 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:18:02.167187 (+ 645us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:02.167206 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:02.168395 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:02.168399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:02.170016 (+ 1617us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:02.170033 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:02.170853 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:02.170863 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:02.171186 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:02.171905 (+ 719us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:18:02.171942 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:18:02.174890 (+ 2948us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:18:02.179064 (+ 4174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:18:02.179074 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:18:02.179091 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:18:02.179661 (+ 570us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:18:02.180103 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:18:02.180110 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:18:02.180116 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:18:02.180332 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:18:02.181032 (+ 700us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:18:02.181041 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:18:02.181433 (+ 392us) client_negotiation.cc:769] Sending connection context
0809 12:18:02.181712 (+ 279us) client_negotiation.cc:240] Negotiation successful
0809 12:18:02.182233 (+ 521us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":274,"threads_started":1}
I20260809 12:18:02.185096 3218 heartbeater.cc:347] Connected to a master server at 127.19.183.254:40669
I20260809 12:18:02.185416 3218 heartbeater.cc:464] Registering TS with master...
I20260809 12:18:02.186331 3218 heartbeater.cc:511] Master 127.19.183.254:40669 requested a full tablet report, sending...
I20260809 12:18:02.187971 2752 ts_manager.cc:194] Registered new tserver with Master: f6fafaec836a4b6ea56041f9757742ce (127.19.183.194:38501)
I20260809 12:18:02.188807 2752 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:57859
WARNING: no policy specified for kudu/127.19.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
I20260809 12:18:02.235389 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40669
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:33821
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:18:02.570693 3229 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:18:02.571089 3229 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:18:02.571249 3229 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:18:02.578343 3229 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:18:02.578490 3229 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:18:02.578668 3229 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:18:02.588176 3229 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:33821
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40669
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:18:02.589461 3229 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:18:02.590831 3229 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:18:02.603724 3235 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:18:02.604012 3236 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:18:02.604188 3239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:18:02.604933 3229 server_base.cc:1034] running on GCE node
I20260809 12:18:02.605523 3229 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:18:02.606379 3229 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:18:02.607673 3229 hybrid_clock.cc:648] HybridClock initialized: now 1786277882607593 us; error 41 us; skew 500 ppm
I20260809 12:18:02.607986 3229 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:18:02 dist-test-slave-39l8 krb5kdc[2683](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277882, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:18:02.611546 3229 init.cc:377] Logged in from keytab as kudu/127.19.183.195@KRBTEST.COM (short username kudu)
I20260809 12:18:02.612906 3229 webserver.cc:460] Webserver started at http://127.19.183.195:36917/ using document root <none> and password file <none>
I20260809 12:18:02.613535 3229 fs_manager.cc:359] Metadata directory not provided
I20260809 12:18:02.613644 3229 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:18:02.613914 3229 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:18:02.617012 3229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "e1d33e7d7c8749eabedefad1eac3ac99"
format_stamp: "Formatted at 2026-08-09 12:18:02 on dist-test-slave-39l8"
server_key: "a07f94daed6c5279c57221feec318b1f"
server_key_iv: "30cfa1259a192edcd99b236fcb3e598d"
server_key_version: "encryptionkey@0"
I20260809 12:18:02.617879 3229 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "e1d33e7d7c8749eabedefad1eac3ac99"
format_stamp: "Formatted at 2026-08-09 12:18:02 on dist-test-slave-39l8"
server_key: "a07f94daed6c5279c57221feec318b1f"
server_key_iv: "30cfa1259a192edcd99b236fcb3e598d"
server_key_version: "encryptionkey@0"
I20260809 12:18:02.623077 3229 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 12:18:02.626520 3253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:18:02.627954 3229 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:18:02.628137 3229 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "e1d33e7d7c8749eabedefad1eac3ac99"
format_stamp: "Formatted at 2026-08-09 12:18:02 on dist-test-slave-39l8"
server_key: "a07f94daed6c5279c57221feec318b1f"
server_key_iv: "30cfa1259a192edcd99b236fcb3e598d"
server_key_version: "encryptionkey@0"
I20260809 12:18:02.628346 3229 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:18:02.648794 3229 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:18:02.651361 3016 heartbeater.cc:503] Master 127.19.183.254:40669 was elected leader, sending a full tablet report...
I20260809 12:18:02.653456 3229 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:18:02.653759 3229 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:18:02.654874 3229 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:18:02.656245 3229 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:18:02.656356 3229 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:18:02.656503 3229 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:18:02.656574 3229 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:18:02.692028 3229 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:39513
I20260809 12:18:02.692063 3444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:39513 every 8 connection(s)
I20260809 12:18:02.693357 3229 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:18:02.702251 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 3229
I20260809 12:18:02.702445 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:18:02.703006 20191 external_mini_cluster.cc:1428] Setting key 8a55bef0c7467853ef580bd4c61ba135
Aug 09 12:18:02 dist-test-slave-39l8 krb5kdc[2683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277882, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:18:02.717599 3448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:02.696212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:56193 (local address 127.19.183.254:40669)
0809 12:18:02.696727 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:02.696732 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:02.698087 (+ 1355us) server_negotiation.cc:407] Connection header received
0809 12:18:02.700091 (+ 2004us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:02.700096 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:02.700207 (+ 111us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:02.700338 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:18:02.701892 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:02.702887 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:02.704918 (+ 2031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:02.705422 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:02.705563 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:02.710235 (+ 4672us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:18:02.710240 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:18:02.710243 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:18:02.710276 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:18:02.713750 (+ 3474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:02.714611 (+ 861us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:02.714623 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:02.714627 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:02.714850 (+ 223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:02.715353 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:02.715358 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:02.715361 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:02.715671 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:18:02.715833 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:02.716736 (+ 903us) server_negotiation.cc:299] Negotiation successful
0809 12:18:02.716967 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":416,"thread_start_us":267,"threads_started":1}
I20260809 12:18:02.718446 3449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:02.696922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:40669 (local address 127.19.183.195:56193)
0809 12:18:02.697873 (+ 951us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:02.697912 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 12:18:02.699749 (+ 1837us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:18:02.700660 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:18:02.700668 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:18:02.701168 (+ 500us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:18:02.701712 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:02.701729 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:02.703054 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:02.703058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:02.704771 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:02.704784 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:02.705673 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:02.705682 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:02.705910 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:02.706870 (+ 960us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:18:02.706905 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:18:02.709990 (+ 3085us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:18:02.713904 (+ 3914us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:18:02.713913 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:18:02.713932 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:18:02.714476 (+ 544us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:18:02.714973 (+ 497us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:18:02.714979 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:18:02.714982 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:18:02.715220 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:18:02.715817 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:18:02.715824 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:18:02.716198 (+ 374us) client_negotiation.cc:769] Sending connection context
0809 12:18:02.716531 (+ 333us) client_negotiation.cc:240] Negotiation successful
0809 12:18:02.717136 (+ 605us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"thread_start_us":336,"threads_started":1}
I20260809 12:18:02.719940 3445 heartbeater.cc:347] Connected to a master server at 127.19.183.254:40669
I20260809 12:18:02.720239 3445 heartbeater.cc:464] Registering TS with master...
I20260809 12:18:02.720965 3445 heartbeater.cc:511] Master 127.19.183.254:40669 requested a full tablet report, sending...
I20260809 12:18:02.722477 2752 ts_manager.cc:194] Registered new tserver with Master: e1d33e7d7c8749eabedefad1eac3ac99 (127.19.183.195:39513)
I20260809 12:18:02.723376 2752 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:56193
I20260809 12:18:02.731192 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:18:02.752568 3448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:02.737056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34370 (local address 127.19.183.254:40669)
0809 12:18:02.737291 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:02.737300 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:02.737915 (+ 615us) server_negotiation.cc:407] Connection header received
0809 12:18:02.738116 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:02.738124 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:02.738217 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:02.738321 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:18:02.740559 (+ 2238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:02.741284 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:02.742950 (+ 1666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:02.743344 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:02.743474 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:02.746185 (+ 2711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:18:02.746189 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:18:02.746193 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:18:02.746223 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:18:02.749298 (+ 3075us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:02.750072 (+ 774us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:02.750079 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:02.750083 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:02.750252 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:02.750669 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:02.750674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:02.750679 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:02.751130 (+ 451us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:18:02.751445 (+ 315us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:02.751834 (+ 389us) server_negotiation.cc:299] Negotiation successful
0809 12:18:02.752164 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 12:18:02.757107 2752 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34370:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:18:02.765684 2752 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:18:02.809154 3467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:02.780535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:38501 (local address 127.0.0.1:32776)
0809 12:18:02.787670 (+ 7135us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:02.787691 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:18:02.788008 (+ 317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:18:02.796274 (+ 8266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:18:02.796282 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:18:02.796315 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:18:02.797119 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:02.797143 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:02.799997 (+ 2854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:02.800003 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:02.804394 (+ 4391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:02.804411 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:02.807419 (+ 3008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:02.807428 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:02.807717 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:02.807772 (+ 55us) client_negotiation.cc:769] Sending connection context
0809 12:18:02.807951 (+ 179us) client_negotiation.cc:240] Negotiation successful
0809 12:18:02.808226 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6976,"mutex_wait_us":1,"thread_start_us":524,"threads_started":1}
I20260809 12:18:02.809194 3472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:02.780204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32776 (local address 127.19.183.194:38501)
0809 12:18:02.795447 (+ 15243us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:02.795461 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:02.795492 (+ 31us) server_negotiation.cc:407] Connection header received
0809 12:18:02.795617 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:02.795629 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:02.795952 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:02.797414 (+ 1462us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:18:02.797680 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:02.799781 (+ 2101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:02.804647 (+ 4866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:02.807105 (+ 2458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:02.807296 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:02.807500 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:02.808339 (+ 839us) server_negotiation.cc:299] Negotiation successful
0809 12:18:02.808597 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15031,"thread_start_us":15455,"threads_started":1}
I20260809 12:18:02.809813 3480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:02.788138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38054 (local address 127.19.183.195:39513)
0809 12:18:02.796397 (+ 8259us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:02.796406 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:02.796440 (+ 34us) server_negotiation.cc:407] Connection header received
0809 12:18:02.796552 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:02.796561 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:02.796855 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:02.797414 (+ 559us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:18:02.798829 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:02.801273 (+ 2444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:02.805569 (+ 4296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:02.807725 (+ 2156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:02.807982 (+ 257us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:02.808198 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:02.808854 (+ 656us) server_negotiation.cc:299] Negotiation successful
0809 12:18:02.809164 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":8076,"thread_start_us":7906,"threads_started":1}
I20260809 12:18:02.811101 3475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:02.788231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:39513 (local address 127.0.0.1:38054)
0809 12:18:02.795402 (+ 7171us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:02.795421 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:18:02.795576 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:18:02.797493 (+ 1917us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:18:02.797499 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:18:02.797531 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:18:02.797993 (+ 462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:02.798009 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:02.801508 (+ 3499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:02.801527 (+ 19us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:02.805301 (+ 3774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:02.805318 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:02.808252 (+ 2934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:02.808261 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:02.808564 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:02.808599 (+ 35us) client_negotiation.cc:769] Sending connection context
0809 12:18:02.810312 (+ 1713us) client_negotiation.cc:240] Negotiation successful
0809 12:18:02.810454 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7046,"mutex_wait_us":1,"thread_start_us":8280,"threads_started":1}
I20260809 12:18:02.811949 3474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:02.779916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:39313 (local address 127.0.0.1:55352)
0809 12:18:02.781514 (+ 1598us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:02.781579 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 12:18:02.781941 (+ 362us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:18:02.800296 (+ 18355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:18:02.800303 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:18:02.800337 (+ 34us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:18:02.801612 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:02.801638 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:02.805520 (+ 3882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:02.805527 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:02.807519 (+ 1992us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:02.807535 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:02.810752 (+ 3217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:02.810760 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:02.811021 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:02.811077 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 12:18:02.811197 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 12:18:02.811343 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1356,"mutex_wait_us":52,"thread_start_us":7291,"threads_started":1}
I20260809 12:18:02.812180 3473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:02.780167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55352 (local address 127.19.183.193:39313)
0809 12:18:02.798866 (+ 18699us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:02.798877 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:02.798908 (+ 31us) server_negotiation.cc:407] Connection header received
0809 12:18:02.799036 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:02.799045 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:02.799256 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:02.799681 (+ 425us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:18:02.801957 (+ 2276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:02.805306 (+ 3349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:02.807755 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:02.810384 (+ 2629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:02.810768 (+ 384us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:02.811010 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:02.811411 (+ 401us) server_negotiation.cc:299] Negotiation successful
0809 12:18:02.811646 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18426,"thread_start_us":19854,"threads_started":1}
I20260809 12:18:02.814638 3333 tablet_service.cc:1467] Processing CreateTablet for tablet 2351dda7be164be184085079ed4ea349 (DEFAULT_TABLE table=test-table [id=6d1c2c7db1194dbfbc4b97c1ec59e40a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:18:02.815270 3118 tablet_service.cc:1467] Processing CreateTablet for tablet 2351dda7be164be184085079ed4ea349 (DEFAULT_TABLE table=test-table [id=6d1c2c7db1194dbfbc4b97c1ec59e40a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:18:02.816251 3333 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2351dda7be164be184085079ed4ea349. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:18:02.815811 2927 tablet_service.cc:1467] Processing CreateTablet for tablet 2351dda7be164be184085079ed4ea349 (DEFAULT_TABLE table=test-table [id=6d1c2c7db1194dbfbc4b97c1ec59e40a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:18:02.816747 3118 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2351dda7be164be184085079ed4ea349. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:18:02.817252 2927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2351dda7be164be184085079ed4ea349. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:18:02.828744 3482 tablet_bootstrap.cc:492] T 2351dda7be164be184085079ed4ea349 P e1d33e7d7c8749eabedefad1eac3ac99: Bootstrap starting.
I20260809 12:18:02.831919 3483 tablet_bootstrap.cc:492] T 2351dda7be164be184085079ed4ea349 P ed3c10a3f08d4b5bbd6fdf04e9443ce0: Bootstrap starting.
I20260809 12:18:02.834745 3482 tablet_bootstrap.cc:654] T 2351dda7be164be184085079ed4ea349 P e1d33e7d7c8749eabedefad1eac3ac99: Neither blocks nor log segments found. Creating new log.
I20260809 12:18:02.836330 3483 tablet_bootstrap.cc:654] T 2351dda7be164be184085079ed4ea349 P ed3c10a3f08d4b5bbd6fdf04e9443ce0: Neither blocks nor log segments found. Creating new log.
I20260809 12:18:02.837033 3482 log.cc:824] T 2351dda7be164be184085079ed4ea349 P e1d33e7d7c8749eabedefad1eac3ac99: Log is configured to *not* fsync() on all Append() calls
I20260809 12:18:02.837873 3483 log.cc:824] T 2351dda7be164be184085079ed4ea349 P ed3c10a3f08d4b5bbd6fdf04e9443ce0: Log is configured to *not* fsync() on all Append() calls
I20260809 12:18:02.837972 3484 tablet_bootstrap.cc:492] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce: Bootstrap starting.
I20260809 12:18:02.840801 3482 tablet_bootstrap.cc:492] T 2351dda7be164be184085079ed4ea349 P e1d33e7d7c8749eabedefad1eac3ac99: No bootstrap required, opened a new log
I20260809 12:18:02.841085 3482 ts_tablet_manager.cc:1397] T 2351dda7be164be184085079ed4ea349 P e1d33e7d7c8749eabedefad1eac3ac99: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.010s
I20260809 12:18:02.841975 3483 tablet_bootstrap.cc:492] T 2351dda7be164be184085079ed4ea349 P ed3c10a3f08d4b5bbd6fdf04e9443ce0: No bootstrap required, opened a new log
I20260809 12:18:02.842159 3484 tablet_bootstrap.cc:654] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce: Neither blocks nor log segments found. Creating new log.
I20260809 12:18:02.842272 3483 ts_tablet_manager.cc:1397] T 2351dda7be164be184085079ed4ea349 P ed3c10a3f08d4b5bbd6fdf04e9443ce0: Time spent bootstrapping tablet: real 0.011s user 0.006s sys 0.002s
I20260809 12:18:02.843336 3484 log.cc:824] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce: Log is configured to *not* fsync() on all Append() calls
I20260809 12:18:02.846344 3484 tablet_bootstrap.cc:492] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce: No bootstrap required, opened a new log
I20260809 12:18:02.846583 3484 ts_tablet_manager.cc:1397] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 12:18:02.850879 3483 raft_consensus.cc:348] T 2351dda7be164be184085079ed4ea349 P ed3c10a3f08d4b5bbd6fdf04e9443ce0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3c10a3f08d4b5bbd6fdf04e9443ce0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 39313 } } peers { permanent_uuid: "f6fafaec836a4b6ea56041f9757742ce" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38501 } } peers { permanent_uuid: "e1d33e7d7c8749eabedefad1eac3ac99" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 39513 } }
I20260809 12:18:02.851251 3483 raft_consensus.cc:374] T 2351dda7be164be184085079ed4ea349 P ed3c10a3f08d4b5bbd6fdf04e9443ce0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:18:02.851382 3483 raft_consensus.cc:729] T 2351dda7be164be184085079ed4ea349 P ed3c10a3f08d4b5bbd6fdf04e9443ce0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed3c10a3f08d4b5bbd6fdf04e9443ce0, State: Initialized, Role: FOLLOWER
I20260809 12:18:02.851490 3482 raft_consensus.cc:348] T 2351dda7be164be184085079ed4ea349 P e1d33e7d7c8749eabedefad1eac3ac99 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3c10a3f08d4b5bbd6fdf04e9443ce0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 39313 } } peers { permanent_uuid: "f6fafaec836a4b6ea56041f9757742ce" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38501 } } peers { permanent_uuid: "e1d33e7d7c8749eabedefad1eac3ac99" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 39513 } }
I20260809 12:18:02.851881 3482 raft_consensus.cc:374] T 2351dda7be164be184085079ed4ea349 P e1d33e7d7c8749eabedefad1eac3ac99 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:18:02.851970 3482 raft_consensus.cc:729] T 2351dda7be164be184085079ed4ea349 P e1d33e7d7c8749eabedefad1eac3ac99 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e1d33e7d7c8749eabedefad1eac3ac99, State: Initialized, Role: FOLLOWER
I20260809 12:18:02.852051 3483 consensus_queue.cc:260] T 2351dda7be164be184085079ed4ea349 P ed3c10a3f08d4b5bbd6fdf04e9443ce0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3c10a3f08d4b5bbd6fdf04e9443ce0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 39313 } } peers { permanent_uuid: "f6fafaec836a4b6ea56041f9757742ce" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38501 } } peers { permanent_uuid: "e1d33e7d7c8749eabedefad1eac3ac99" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 39513 } }
I20260809 12:18:02.852543 3482 consensus_queue.cc:260] T 2351dda7be164be184085079ed4ea349 P e1d33e7d7c8749eabedefad1eac3ac99 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3c10a3f08d4b5bbd6fdf04e9443ce0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 39313 } } peers { permanent_uuid: "f6fafaec836a4b6ea56041f9757742ce" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38501 } } peers { permanent_uuid: "e1d33e7d7c8749eabedefad1eac3ac99" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 39513 } }
I20260809 12:18:02.853988 3483 ts_tablet_manager.cc:1428] T 2351dda7be164be184085079ed4ea349 P ed3c10a3f08d4b5bbd6fdf04e9443ce0: Time spent starting tablet: real 0.012s user 0.010s sys 0.002s
I20260809 12:18:02.855098 3445 heartbeater.cc:503] Master 127.19.183.254:40669 was elected leader, sending a full tablet report...
I20260809 12:18:02.855589 3482 ts_tablet_manager.cc:1428] T 2351dda7be164be184085079ed4ea349 P e1d33e7d7c8749eabedefad1eac3ac99: Time spent starting tablet: real 0.014s user 0.015s sys 0.001s
I20260809 12:18:02.856971 3484 raft_consensus.cc:348] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3c10a3f08d4b5bbd6fdf04e9443ce0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 39313 } } peers { permanent_uuid: "f6fafaec836a4b6ea56041f9757742ce" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38501 } } peers { permanent_uuid: "e1d33e7d7c8749eabedefad1eac3ac99" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 39513 } }
I20260809 12:18:02.857290 3484 raft_consensus.cc:374] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:18:02.857383 3484 raft_consensus.cc:729] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6fafaec836a4b6ea56041f9757742ce, State: Initialized, Role: FOLLOWER
I20260809 12:18:02.858001 3484 consensus_queue.cc:260] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3c10a3f08d4b5bbd6fdf04e9443ce0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 39313 } } peers { permanent_uuid: "f6fafaec836a4b6ea56041f9757742ce" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38501 } } peers { permanent_uuid: "e1d33e7d7c8749eabedefad1eac3ac99" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 39513 } }
I20260809 12:18:02.859652 3484 ts_tablet_manager.cc:1428] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce: Time spent starting tablet: real 0.013s user 0.009s sys 0.005s
I20260809 12:18:02.859881 3218 heartbeater.cc:503] Master 127.19.183.254:40669 was elected leader, sending a full tablet report...
W20260809 12:18:02.869410 3017 tablet.cc:2369] T 2351dda7be164be184085079ed4ea349 P ed3c10a3f08d4b5bbd6fdf04e9443ce0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:18:02.911409 3220 tablet.cc:2369] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:18:02.945405 3446 tablet.cc:2369] T 2351dda7be164be184085079ed4ea349 P e1d33e7d7c8749eabedefad1eac3ac99: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:18:03.031281 3495 raft_consensus.cc:482] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:18:03.031615 3495 raft_consensus.cc:504] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3c10a3f08d4b5bbd6fdf04e9443ce0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 39313 } } peers { permanent_uuid: "f6fafaec836a4b6ea56041f9757742ce" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38501 } } peers { permanent_uuid: "e1d33e7d7c8749eabedefad1eac3ac99" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 39513 } }
I20260809 12:18:03.033526 3495 leader_election.cc:288] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ed3c10a3f08d4b5bbd6fdf04e9443ce0 (127.19.183.193:39313), e1d33e7d7c8749eabedefad1eac3ac99 (127.19.183.195:39513)
I20260809 12:18:03.041368 3473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:03.033975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:59581 (local address 127.19.183.193:39313)
0809 12:18:03.034271 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:03.034281 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:03.034778 (+ 497us) server_negotiation.cc:407] Connection header received
0809 12:18:03.035176 (+ 398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:03.035180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:03.035245 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:03.035296 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:18:03.036042 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:03.037155 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:03.039236 (+ 2081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:03.040259 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:03.040484 (+ 225us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:03.040574 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:03.040897 (+ 323us) server_negotiation.cc:299] Negotiation successful
0809 12:18:03.041031 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 12:18:03.041487 3496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:03.033911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:39313 (local address 127.19.183.194:59581)
0809 12:18:03.034620 (+ 709us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:03.034644 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:18:03.034915 (+ 271us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:18:03.035468 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:18:03.035471 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:18:03.035492 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:18:03.035888 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:03.035898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:03.037326 (+ 1428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:03.037333 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:03.039054 (+ 1721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:03.039068 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:03.040551 (+ 1483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:03.040558 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:03.040705 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:03.040730 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 12:18:03.040852 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 12:18:03.040993 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":573,"mutex_wait_us":1,"thread_start_us":406,"threads_started":1}
I20260809 12:18:03.043025 2954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2351dda7be164be184085079ed4ea349" candidate_uuid: "f6fafaec836a4b6ea56041f9757742ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed3c10a3f08d4b5bbd6fdf04e9443ce0" is_pre_election: true
I20260809 12:18:03.043715 2954 raft_consensus.cc:2454] T 2351dda7be164be184085079ed4ea349 P ed3c10a3f08d4b5bbd6fdf04e9443ce0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6fafaec836a4b6ea56041f9757742ce in term 0.
I20260809 12:18:03.044580 3050 leader_election.cc:302] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ed3c10a3f08d4b5bbd6fdf04e9443ce0, f6fafaec836a4b6ea56041f9757742ce; no voters:
I20260809 12:18:03.045131 3495 raft_consensus.cc:2781] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:18:03.045357 3495 raft_consensus.cc:482] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:18:03.045547 3495 raft_consensus.cc:3037] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:18:03.045615 3480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:03.034890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:42399 (local address 127.19.183.195:39513)
0809 12:18:03.035090 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:03.035113 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:03.036773 (+ 1660us) server_negotiation.cc:407] Connection header received
0809 12:18:03.036871 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:03.036878 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:03.036992 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:03.037125 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:18:03.037903 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:03.039085 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:03.042560 (+ 3475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:03.044269 (+ 1709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:03.044484 (+ 215us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:03.044579 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:03.045159 (+ 580us) server_negotiation.cc:299] Negotiation successful
0809 12:18:03.045312 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 12:18:03.046226 3500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:03.034743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:39513 (local address 127.19.183.194:42399)
0809 12:18:03.036622 (+ 1879us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:03.036640 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:18:03.036766 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:18:03.037171 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:18:03.037175 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:18:03.037205 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:18:03.037700 (+ 495us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:03.037715 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:03.039265 (+ 1550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:03.039273 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:03.042332 (+ 3059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:03.042352 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:03.044593 (+ 2241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:03.044602 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:03.044929 (+ 327us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:03.044981 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 12:18:03.045086 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 12:18:03.045248 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1744,"thread_start_us":1528,"threads_started":1}
I20260809 12:18:03.047657 3366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2351dda7be164be184085079ed4ea349" candidate_uuid: "f6fafaec836a4b6ea56041f9757742ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1d33e7d7c8749eabedefad1eac3ac99" is_pre_election: true
I20260809 12:18:03.048295 3366 raft_consensus.cc:2454] T 2351dda7be164be184085079ed4ea349 P e1d33e7d7c8749eabedefad1eac3ac99 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6fafaec836a4b6ea56041f9757742ce in term 0.
I20260809 12:18:03.049701 3495 raft_consensus.cc:504] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3c10a3f08d4b5bbd6fdf04e9443ce0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 39313 } } peers { permanent_uuid: "f6fafaec836a4b6ea56041f9757742ce" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38501 } } peers { permanent_uuid: "e1d33e7d7c8749eabedefad1eac3ac99" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 39513 } }
I20260809 12:18:03.050307 3495 leader_election.cc:288] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce [CANDIDATE]: Term 1 election: Requested vote from peers ed3c10a3f08d4b5bbd6fdf04e9443ce0 (127.19.183.193:39313), e1d33e7d7c8749eabedefad1eac3ac99 (127.19.183.195:39513)
I20260809 12:18:03.050822 3366 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2351dda7be164be184085079ed4ea349" candidate_uuid: "f6fafaec836a4b6ea56041f9757742ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e1d33e7d7c8749eabedefad1eac3ac99"
I20260809 12:18:03.050823 2954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2351dda7be164be184085079ed4ea349" candidate_uuid: "f6fafaec836a4b6ea56041f9757742ce" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ed3c10a3f08d4b5bbd6fdf04e9443ce0"
I20260809 12:18:03.050998 3366 raft_consensus.cc:3037] T 2351dda7be164be184085079ed4ea349 P e1d33e7d7c8749eabedefad1eac3ac99 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:18:03.051056 2954 raft_consensus.cc:3037] T 2351dda7be164be184085079ed4ea349 P ed3c10a3f08d4b5bbd6fdf04e9443ce0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:18:03.053249 3366 raft_consensus.cc:2454] T 2351dda7be164be184085079ed4ea349 P e1d33e7d7c8749eabedefad1eac3ac99 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6fafaec836a4b6ea56041f9757742ce in term 1.
I20260809 12:18:03.053658 3050 leader_election.cc:302] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e1d33e7d7c8749eabedefad1eac3ac99, f6fafaec836a4b6ea56041f9757742ce; no voters:
I20260809 12:18:03.054051 3495 raft_consensus.cc:2781] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:18:03.054736 3495 raft_consensus.cc:686] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce [term 1 LEADER]: Becoming Leader. State: Replica: f6fafaec836a4b6ea56041f9757742ce, State: Running, Role: LEADER
I20260809 12:18:03.054772 2954 raft_consensus.cc:2454] T 2351dda7be164be184085079ed4ea349 P ed3c10a3f08d4b5bbd6fdf04e9443ce0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6fafaec836a4b6ea56041f9757742ce in term 1.
I20260809 12:18:03.055390 3495 consensus_queue.cc:237] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3c10a3f08d4b5bbd6fdf04e9443ce0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 39313 } } peers { permanent_uuid: "f6fafaec836a4b6ea56041f9757742ce" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38501 } } peers { permanent_uuid: "e1d33e7d7c8749eabedefad1eac3ac99" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 39513 } }
I20260809 12:18:03.060489 2750 catalog_manager.cc:5495] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce reported cstate change: term changed from 0 to 1, leader changed from <none> to f6fafaec836a4b6ea56041f9757742ce (127.19.183.194). New cstate: current_term: 1 leader_uuid: "f6fafaec836a4b6ea56041f9757742ce" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ed3c10a3f08d4b5bbd6fdf04e9443ce0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 39313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6fafaec836a4b6ea56041f9757742ce" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38501 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e1d33e7d7c8749eabedefad1eac3ac99" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 39513 } health_report { overall_health: UNKNOWN } } }
I20260809 12:18:03.091467 3472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:03.085752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32792 (local address 127.19.183.194:38501)
0809 12:18:03.085941 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:03.085946 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:03.085964 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:18:03.086056 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:03.086059 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:03.086147 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:03.086282 (+ 135us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:18:03.086699 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:03.087467 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:03.089157 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:03.089544 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:03.089644 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:03.090222 (+ 578us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:18:03.090813 (+ 591us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:18:03.090977 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:03.091108 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 12:18:03.091173 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 12:18:03.099444 3366 raft_consensus.cc:1261] T 2351dda7be164be184085079ed4ea349 P e1d33e7d7c8749eabedefad1eac3ac99 [term 1 FOLLOWER]: Refusing update from remote peer f6fafaec836a4b6ea56041f9757742ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:18:03.099582 2954 raft_consensus.cc:1261] T 2351dda7be164be184085079ed4ea349 P ed3c10a3f08d4b5bbd6fdf04e9443ce0 [term 1 FOLLOWER]: Refusing update from remote peer f6fafaec836a4b6ea56041f9757742ce: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:18:03.100492 3501 consensus_queue.cc:1035] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "e1d33e7d7c8749eabedefad1eac3ac99" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 39513 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:18:03.100891 3495 consensus_queue.cc:1035] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce [LEADER]: Connected to new peer: Peer: permanent_uuid: "ed3c10a3f08d4b5bbd6fdf04e9443ce0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 39313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:18:03.134853 2750 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:18:03.135108 2750 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34370:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:18:03.138617 2750 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8119f296c59d40ee833e7ebfb25cdfcd: Sending DeleteTablet for 3 replicas of tablet 2351dda7be164be184085079ed4ea349
I20260809 12:18:03.139851 2927 tablet_service.cc:1514] Processing DeleteTablet for tablet 2351dda7be164be184085079ed4ea349 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:18:03 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:55352
I20260809 12:18:03.140034 3118 tablet_service.cc:1514] Processing DeleteTablet for tablet 2351dda7be164be184085079ed4ea349 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:18:03 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:32776
I20260809 12:18:03.140162 3333 tablet_service.cc:1514] Processing DeleteTablet for tablet 2351dda7be164be184085079ed4ea349 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:18:03 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:38054
I20260809 12:18:03.142290 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 2833
I20260809 12:18:03.152786 3521 tablet_replica.cc:330] T 2351dda7be164be184085079ed4ea349 P ed3c10a3f08d4b5bbd6fdf04e9443ce0: stopping tablet replica
I20260809 12:18:03.153509 3521 raft_consensus.cc:2229] T 2351dda7be164be184085079ed4ea349 P ed3c10a3f08d4b5bbd6fdf04e9443ce0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:18:03.154352 3521 raft_consensus.cc:2258] T 2351dda7be164be184085079ed4ea349 P ed3c10a3f08d4b5bbd6fdf04e9443ce0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:18:03.157203 3521 ts_tablet_manager.cc:1905] T 2351dda7be164be184085079ed4ea349 P ed3c10a3f08d4b5bbd6fdf04e9443ce0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:18:03.164841 3521 ts_tablet_manager.cc:1918] T 2351dda7be164be184085079ed4ea349 P ed3c10a3f08d4b5bbd6fdf04e9443ce0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:18:03.165082 3521 log.cc:1196] T 2351dda7be164be184085079ed4ea349 P ed3c10a3f08d4b5bbd6fdf04e9443ce0: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/2351dda7be164be184085079ed4ea349
I20260809 12:18:03.165699 3521 ts_tablet_manager.cc:1939] T 2351dda7be164be184085079ed4ea349 P ed3c10a3f08d4b5bbd6fdf04e9443ce0: Deleting consensus metadata
Aug 09 12:18:03 dist-test-slave-39l8 krb5kdc[2683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.193@KRBTEST.COM
I20260809 12:18:03.167578 2735 catalog_manager.cc:4841] TS ed3c10a3f08d4b5bbd6fdf04e9443ce0 (127.19.183.193:39313): tablet 2351dda7be164be184085079ed4ea349 (table test-table [id=6d1c2c7db1194dbfbc4b97c1ec59e40a]) successfully deleted
I20260809 12:18:03.170945 3523 tablet_replica.cc:330] T 2351dda7be164be184085079ed4ea349 P e1d33e7d7c8749eabedefad1eac3ac99: stopping tablet replica
I20260809 12:18:03.171535 3523 raft_consensus.cc:2229] T 2351dda7be164be184085079ed4ea349 P e1d33e7d7c8749eabedefad1eac3ac99 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:18:03.172155 3523 raft_consensus.cc:2258] T 2351dda7be164be184085079ed4ea349 P e1d33e7d7c8749eabedefad1eac3ac99 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:18:03.174237 3523 ts_tablet_manager.cc:1905] T 2351dda7be164be184085079ed4ea349 P e1d33e7d7c8749eabedefad1eac3ac99: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:18:03.177527 3473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:03.143184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55358 (local address 127.19.183.193:39313)
0809 12:18:03.143359 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:03.143367 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:03.148648 (+ 5281us) server_negotiation.cc:407] Connection header received
0809 12:18:03.149828 (+ 1180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:03.149834 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:03.149919 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:03.149997 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:18:03.153721 (+ 3724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:03.155024 (+ 1303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:03.158783 (+ 3759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:03.159510 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:03.159615 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:03.170113 (+ 10498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:18:03.170138 (+ 25us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:18:03.170155 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:18:03.170201 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:18:03.173940 (+ 3739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:03.174864 (+ 924us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:03.174873 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:03.174877 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:03.175045 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:03.175533 (+ 488us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:03.175538 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:03.175542 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:03.176002 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:18:03.176299 (+ 297us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:03.176895 (+ 596us) server_negotiation.cc:299] Negotiation successful
0809 12:18:03.177143 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 12:18:03.179080 3522 tablet_replica.cc:330] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce: stopping tablet replica
I20260809 12:18:03.179271 3009 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:18:03.179557 3522 raft_consensus.cc:2229] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:18:03.180500 3522 raft_consensus.cc:2258] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:18:03.182847 3522 ts_tablet_manager.cc:1905] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:18:03.187798 3523 ts_tablet_manager.cc:1918] T 2351dda7be164be184085079ed4ea349 P e1d33e7d7c8749eabedefad1eac3ac99: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:18:03.190531 3523 log.cc:1196] T 2351dda7be164be184085079ed4ea349 P e1d33e7d7c8749eabedefad1eac3ac99: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/2351dda7be164be184085079ed4ea349
I20260809 12:18:03.190788 3522 ts_tablet_manager.cc:1918] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:18:03.191028 3522 log.cc:1196] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/2351dda7be164be184085079ed4ea349
I20260809 12:18:03.191457 3523 ts_tablet_manager.cc:1939] T 2351dda7be164be184085079ed4ea349 P e1d33e7d7c8749eabedefad1eac3ac99: Deleting consensus metadata
I20260809 12:18:03.191622 3522 ts_tablet_manager.cc:1939] T 2351dda7be164be184085079ed4ea349 P f6fafaec836a4b6ea56041f9757742ce: Deleting consensus metadata
I20260809 12:18:03.193672 2737 catalog_manager.cc:4841] TS f6fafaec836a4b6ea56041f9757742ce (127.19.183.194:38501): tablet 2351dda7be164be184085079ed4ea349 (table test-table [id=6d1c2c7db1194dbfbc4b97c1ec59e40a]) successfully deleted
I20260809 12:18:03.193795 2735 catalog_manager.cc:4841] TS e1d33e7d7c8749eabedefad1eac3ac99 (127.19.183.195:39513): tablet 2351dda7be164be184085079ed4ea349 (table test-table [id=6d1c2c7db1194dbfbc4b97c1ec59e40a]) successfully deleted
I20260809 12:18:03.281409 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 2833
I20260809 12:18:03.316558 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 3025
Aug 09 12:18:03 dist-test-slave-39l8 krb5kdc[2683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.194@KRBTEST.COM
I20260809 12:18:03.334048 3472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:03.317362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32806 (local address 127.19.183.194:38501)
0809 12:18:03.317533 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:03.317539 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:03.317560 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:18:03.317742 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:03.317748 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:03.317840 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:03.317946 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:18:03.319383 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:03.320644 (+ 1261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:03.321871 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:03.322565 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:03.322697 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:03.326984 (+ 4287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:18:03.327004 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:18:03.327026 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:18:03.327092 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:18:03.331110 (+ 4018us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:03.331725 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:03.331735 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:03.331739 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:03.331954 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:03.332344 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:03.332351 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:03.332356 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:03.332837 (+ 481us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:18:03.333201 (+ 364us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:03.333429 (+ 228us) server_negotiation.cc:299] Negotiation successful
0809 12:18:03.333655 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 12:18:03.334808 3212 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:18:03.448352 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 3025
I20260809 12:18:03.481393 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 3229
Aug 09 12:18:03 dist-test-slave-39l8 krb5kdc[2683](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277880, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.195@KRBTEST.COM
I20260809 12:18:03.495432 3480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:03.482154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38056 (local address 127.19.183.195:39513)
0809 12:18:03.482329 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:03.482333 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:03.482387 (+ 54us) server_negotiation.cc:407] Connection header received
0809 12:18:03.482539 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:03.482543 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:03.482607 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:03.482694 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:18:03.484030 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:03.484726 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:03.485864 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:03.486235 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:03.486322 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:03.490431 (+ 4109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:18:03.490444 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:18:03.490459 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:18:03.490489 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:18:03.493096 (+ 2607us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:03.493668 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:03.493674 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:03.493677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:03.493810 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:03.494130 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:03.494133 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:03.494135 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:03.494455 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:18:03.494690 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:03.495016 (+ 326us) server_negotiation.cc:299] Negotiation successful
0809 12:18:03.495185 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 12:18:03.496055 3439 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:18:03.583236 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 3229
I20260809 12:18:03.608312 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 2705
I20260809 12:18:03.609580 2792 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:18:03.719733 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 2705
2026-08-09T12:18:03Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (4799 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:18:03 dist-test-slave-39l8 krb5kdc[3534](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:18:03 dist-test-slave-39l8 krb5kdc[3534](info): set up 2 sockets
Aug 09 12:18:03 dist-test-slave-39l8 krb5kdc[3534](info): commencing operation
krb5kdc: starting...
W20260809 12:18:05.065618 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.292s user 0.000s sys 0.039s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:18:05 dist-test-slave-39l8 krb5kdc[3534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277885, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:18:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:18:05Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.19.183.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.19.183.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.19.183.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/rangeradmin_127.19.183.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.19.183.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/rangeradmin_127.19.183.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.19.183.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.19.183.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.19.183.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/rangerlookup_127.19.183.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.19.183.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/rangerlookup_127.19.183.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.183.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.19.183.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.183.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/HTTP_127.19.183.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.183.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/HTTP_127.19.183.212@KRBTEST.COM.keytab.
I20260809 12:18:05.465734 20191 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskqUnAOm/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/postgres -l logfile start
2026-08-09 12:18:07.836 UTC [3580] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 12:18:07.836 UTC [3580] LOG: listening on IPv4 address "127.19.183.212", port 47605
2026-08-09 12:18:07.839 UTC [3580] LOG: listening on Unix socket "/tmp/.s.PGSQL.47605"
2026-08-09 12:18:07.860 UTC [3586] LOG: database system was shut down at 2026-08-09 12:18:06 UTC
2026-08-09 12:18:07.863 UTC [3580] LOG: database system is ready to accept connections
I20260809 12:18:09.043702 20191 mini_postgres.cc:96] Postgres bound to 47605
2026-08-09 12:18:09.072 UTC [3598] FATAL: database "slave" does not exist
127.19.183.212:47605 - accepting connections
I20260809 12:18:09.073061 20191 mini_ranger.cc:161] Starting Ranger...
I20260809 12:18:09.108572 20191 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 12:18:09.571378 20191 mini_ranger.cc:87] Created ranger Postgres database
I20260809 12:18:09.571581 20191 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ranger-admin
2026-08-09 12:18:09,961 [I] DB FLAVOR :POSTGRES
2026-08-09 12:18:09,961 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 12:18:09,961 [I] Checking connection..
2026-08-09 12:18:09,961 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:47605/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 12:18:10,261 [I] Checking connection passed.
2026-08-09 12:18:10,262 [I] --------- Verifying version history table ---------
2026-08-09 12:18:10,262 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:47605/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 12:18:10,538 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 12:18:10,538 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:47605/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 12:18:10,782 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 12:18:10,782 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 12:18:10,783 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:47605/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 12:18:11.033 UTC [3692] WARNING: there is no transaction in progress
2026-08-09 12:18:11,043 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 12:18:11,043 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:47605/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 12:18:11,327 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 12:18:11,327 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 12:18:11,327 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:47605/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 12:18:11,598 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:47605/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 12:18:11,866 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:47605/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-39l8.c.gcp-upstream.internal','N') ;"
2026-08-09 12:18:12,110 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:47605/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 12:18:12,389 [I] Table x_portal_user does not exist in database ranger
2026-08-09 12:18:12,389 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:47605/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 12:18:12,676 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 12:18:12,677 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:47605/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 12:18:12,920 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:47605/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 12:18:13,202 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 12:18:13,203 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:47605/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 12:18:14.223 UTC [3867] WARNING: there is no transaction in progress
2026-08-09 12:18:14.243 UTC [3867] WARNING: there is no transaction in progress
2026-08-09 12:18:14.261 UTC [3867] WARNING: there is no transaction in progress
2026-08-09 12:18:14.278 UTC [3867] WARNING: there is no transaction in progress
2026-08-09 12:18:14.298 UTC [3867] WARNING: there is no transaction in progress
2026-08-09 12:18:14.321 UTC [3867] WARNING: there is no transaction in progress
2026-08-09 12:18:14.383 UTC [3867] WARNING: there is no transaction in progress
2026-08-09 12:18:14.391 UTC [3867] WARNING: there is no transaction in progress
2026-08-09 12:18:14.405 UTC [3867] WARNING: there is no transaction in progress
2026-08-09 12:18:14.415 UTC [3867] WARNING: there is no transaction in progress
2026-08-09 12:18:14.427 UTC [3867] WARNING: there is no transaction in progress
2026-08-09 12:18:14.441 UTC [3867] WARNING: there is no transaction in progress
2026-08-09 12:18:14.451 UTC [3867] WARNING: there is no transaction in progress
2026-08-09 12:18:14.458 UTC [3867] WARNING: there is no transaction in progress
2026-08-09 12:18:14.464 UTC [3867] WARNING: there is no transaction in progress
2026-08-09 12:18:14.470 UTC [3867] WARNING: there is no transaction in progress
2026-08-09 12:18:15,100 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 12:18:15,101 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:47605/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 12:18:15,379 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 12:18:15,379 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:47605/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 12:18:15,648 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 12:18:15,648 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:47605/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 12:18:15,932 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 12:18:15,933 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:47605/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 12:18:16,245 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:47605/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 12:18:16,508 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:47605/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-39l8.c.gcp-upstream.internal';"
2026-08-09 12:18:16,745 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 12:18:16,745 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:47605/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 12:18:17,002 [I] DB_PATCHES have already been applied
I20260809 12:18:17.008510 20191 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ranger-admin:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskqUnAOm/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 12:18:17.008692 20191 mini_ranger.cc:193] Using host: 127.19.183.212
I20260809 12:18:17.035856 20191 mini_ranger.cc:227] Ranger admin URL: http://127.19.183.212:49169
Aug 09, 2026 12:18:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:18:17 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:18:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 12:18:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 12:18:17 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 12:18:18 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 12:18:18 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 12:18:18.448514 20191 mini_ranger.cc:160] Time spent starting Ranger: real 9.375s user 0.004s sys 0.075s
Aug 09, 2026 12:18:39 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 12:18:39 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 09, 2026 12:18:39 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 12:18:39 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 12:18:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 12:18:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 12:18:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 12:18:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 12:18:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 12:18:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 12:18:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 12:18:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 12:18:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 12:18:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 12:18:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 12:18:39 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 12:18:39 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 12:18:40 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 12:18:40.681902 20191 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.19.183.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.19.183.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.19.183.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/rangerkms_127.19.183.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.19.183.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/rangerkms_127.19.183.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.183.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.19.183.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.183.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/HTTP_127.19.183.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.183.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/HTTP_127.19.183.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 12:18:40 dist-test-slave-39l8 krb5kdc[3534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277920, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 12:18:40.917037 20191 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 12:18:40.947541 20191 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 12:18:41.671459 20191 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 12:18:41.671609 20191 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ranger-kms
I20260809 12:18:41.671775 20191 mini_ranger_kms.cc:227] Using postgres at 127.19.183.212:47605
2026-08-09 12:18:42,476 [I] DB FLAVOR :POSTGRES
2026-08-09 12:18:42,476 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 12:18:42,476 [I] Checking connection
2026-08-09 12:18:42,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:47605/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 12:18:42,727 [I] connection success
2026-08-09 12:18:42,727 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 12:18:42,727 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 12:18:42,727 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:47605/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 12:18:42,960 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 12:18:42,960 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 12:18:42,960 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 12:18:42,960 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:47605/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 12:18:43,232 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 12:18:43.732647 20191 mini_ranger_kms.cc:310] Created kms service
I20260809 12:18:43.938657 20191 mini_ranger_kms.cc:326] Created kudu user
I20260809 12:18:43.984700 20191 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 12:18:43 dist-test-slave-39l8 krb5kdc[3534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277920, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
I20260809 12:18:44.411870 20191 mini_ranger_kms.cc:384] Added ranger policy
I20260809 12:18:44.412309 20191 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ranger-kms:/tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 12:18:44.412385 20191 mini_ranger_kms.cc:240] Using host: 127.19.183.212
I20260809 12:18:44.446870 20191 mini_ranger_kms.cc:276] Ranger KMS PID: 4139
I20260809 12:18:44.447067 20191 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.19.183.212:55665
Aug 09, 2026 12:18:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:18:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:18:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 12:18:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 12:18:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 12:18:45 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 12:18:46 dist-test-slave-39l8 krb5kdc[3534](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786277926, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.183.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:18:47.173374 20191 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.256s user 0.152s sys 0.448s
I20260809 12:18:47.173763 20191 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 12:18:47.174001 20191 mini_ranger_kms.cc:401] 127.19.183.212:55665/kms/v1/keys
Aug 09 12:18:47 dist-test-slave-39l8 krb5kdc[3534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277926, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.183.212@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
Aug 09 12:18:47 dist-test-slave-39l8 krb5kdc[3534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277926, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.183.212@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
Aug 09 12:18:50 dist-test-slave-39l8 krb5kdc[3534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:18:50.288322 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:38693
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:45583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:38693
--ranger_config_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.183.212:55665/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:18:50.909895 4216 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:18:50.910259 4216 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:18:50.910415 4216 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:18:50.920141 4216 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:18:50.920312 4216 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:18:50.920413 4216 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:18:50.920557 4216 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:18:50.920673 4216 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:18:50.932893 4216 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:45583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/master-0/wal
--ranger_kms_url=127.19.183.212:55665/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:38693
--ranger_config_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:38693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:18:50.934350 4216 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:18:50.935963 4216 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:18:50.953838 4223 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:18:50.953673 4226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:18:50.953896 4224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:18:50.955485 4216 server_base.cc:1034] running on GCE node
I20260809 12:18:50.956460 4216 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:18:50.958099 4216 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:18:50.959406 4216 hybrid_clock.cc:648] HybridClock initialized: now 1786277930959287 us; error 31 us; skew 500 ppm
I20260809 12:18:50.959905 4216 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:18:50 dist-test-slave-39l8 krb5kdc[3534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277930, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:18:50.964021 4216 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:18:50.965858 4216 webserver.cc:460] Webserver started at http://127.19.183.254:42061/ using document root <none> and password file <none>
I20260809 12:18:50.966828 4216 fs_manager.cc:359] Metadata directory not provided
I20260809 12:18:50.966979 4216 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:18:50.967273 4216 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:18:50 dist-test-slave-39l8 krb5kdc[3534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277930, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
I20260809 12:18:51.041575 4216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "0773e4c8c8ea4d3583615cb85ac297dd"
format_stamp: "Formatted at 2026-08-09 12:18:51 on dist-test-slave-39l8"
server_key: "0676426497e6c5cf70d241d7bdff8fea"
server_key_iv: "cbc3d7c6531a580627555747c36b065d"
server_key_version: "kuduclusterkey@0"
I20260809 12:18:51.042692 4216 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "0773e4c8c8ea4d3583615cb85ac297dd"
format_stamp: "Formatted at 2026-08-09 12:18:51 on dist-test-slave-39l8"
server_key: "0676426497e6c5cf70d241d7bdff8fea"
server_key_iv: "cbc3d7c6531a580627555747c36b065d"
server_key_version: "kuduclusterkey@0"
I20260809 12:18:51.050555 4216 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.005s
I20260809 12:18:51.073098 4234 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:18:51.074922 4216 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.000s
I20260809 12:18:51.075168 4216 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "0773e4c8c8ea4d3583615cb85ac297dd"
format_stamp: "Formatted at 2026-08-09 12:18:51 on dist-test-slave-39l8"
server_key: "0676426497e6c5cf70d241d7bdff8fea"
server_key_iv: "cbc3d7c6531a580627555747c36b065d"
server_key_version: "kuduclusterkey@0"
I20260809 12:18:51.075425 4216 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:18:51.100458 4216 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:18:51.106123 4216 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:18:51.106604 4216 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:18:51.150540 4216 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:38693
I20260809 12:18:51.150687 4314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:38693 every 8 connection(s)
I20260809 12:18:51.152391 4216 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:18:51.158864 4315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:18:51.160146 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 4216
I20260809 12:18:51.160431 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:18:51.161860 20191 external_mini_cluster.cc:1428] Setting key 2c5c684ebdccefe55af86bfd97d5a5c0
I20260809 12:18:51.179010 4315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd: Bootstrap starting.
Aug 09 12:18:51 dist-test-slave-39l8 krb5kdc[3534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:18:51.184290 4315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd: Neither blocks nor log segments found. Creating new log.
I20260809 12:18:51.186326 4315 log.cc:824] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd: Log is configured to *not* fsync() on all Append() calls
I20260809 12:18:51.191325 4315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd: No bootstrap required, opened a new log
I20260809 12:18:51.192129 4322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:51.165400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56034 (local address 127.19.183.254:38693)
0809 12:18:51.167315 (+ 1915us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:51.167327 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:51.167388 (+ 61us) server_negotiation.cc:407] Connection header received
0809 12:18:51.170530 (+ 3142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:51.170556 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:51.171051 (+ 495us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:51.171500 (+ 449us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:18:51.173008 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:51.174914 (+ 1906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:51.176236 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:51.177176 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:51.177370 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:51.182154 (+ 4784us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:18:51.182163 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:18:51.182176 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:18:51.182213 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:18:51.187046 (+ 4833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:51.187979 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:51.187998 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:51.188015 (+ 17us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:51.188345 (+ 330us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:51.188864 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:51.188868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:51.188870 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:51.189491 (+ 621us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:18:51.189858 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:51.190419 (+ 561us) server_negotiation.cc:299] Negotiation successful
0809 12:18:51.190790 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1609,"thread_start_us":1081,"threads_started":1}
I20260809 12:18:51.205264 4315 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0773e4c8c8ea4d3583615cb85ac297dd" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38693 } }
I20260809 12:18:51.205858 4315 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:18:51.206089 4315 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0773e4c8c8ea4d3583615cb85ac297dd, State: Initialized, Role: FOLLOWER
I20260809 12:18:51.206907 4315 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0773e4c8c8ea4d3583615cb85ac297dd" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38693 } }
I20260809 12:18:51.207255 4315 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:18:51.207543 4315 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:18:51.207871 4315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:18:51.211604 4315 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0773e4c8c8ea4d3583615cb85ac297dd" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38693 } }
I20260809 12:18:51.212271 4315 leader_election.cc:302] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0773e4c8c8ea4d3583615cb85ac297dd; no voters:
I20260809 12:18:51.213649 4315 leader_election.cc:288] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:18:51.214185 4324 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:18:51.216482 4324 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd [term 1 LEADER]: Becoming Leader. State: Replica: 0773e4c8c8ea4d3583615cb85ac297dd, State: Running, Role: LEADER
I20260809 12:18:51.217608 4324 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0773e4c8c8ea4d3583615cb85ac297dd" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38693 } }
I20260809 12:18:51.218226 4315 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:18:51.223474 4327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0773e4c8c8ea4d3583615cb85ac297dd. Latest consensus state: current_term: 1 leader_uuid: "0773e4c8c8ea4d3583615cb85ac297dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0773e4c8c8ea4d3583615cb85ac297dd" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38693 } } }
I20260809 12:18:51.223893 4325 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0773e4c8c8ea4d3583615cb85ac297dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0773e4c8c8ea4d3583615cb85ac297dd" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38693 } } }
I20260809 12:18:51.226468 4325 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd [sys.catalog]: This master's current role is: LEADER
I20260809 12:18:51.226468 4327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd [sys.catalog]: This master's current role is: LEADER
I20260809 12:18:51.230897 4336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:18:51.241945 4336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:18:51.243861 4315 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 12:18:51.287061 4336 catalog_manager.cc:1347] Generated new cluster ID: cd5f2e7dd5d04abfafb6ab0dcdd4aa4a
I20260809 12:18:51.287355 4336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:18:51.308571 4336 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:18:51.310824 4336 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:18:51.321800 4336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd: Generated new TSK 0
I20260809 12:18:51.322932 4336 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
I20260809 12:18:52.239502 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.183.212:55665/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:38693
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:45583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:18:52 dist-test-slave-39l8 krb5kdc[3534](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786277932, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 12:18:52.905392 4388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:18:52.905766 4388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:18:52.906047 4388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:18:52.920464 4388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:18:52.920718 4388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:18:52.920951 4388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:18:52.934202 4388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:45583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.19.183.212:55665/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:38693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:18:52.936017 4388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:18:52.937820 4388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:18:52.966296 4405 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:18:52.967830 4403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:18:52.977910 4388 server_base.cc:1034] running on GCE node
W20260809 12:18:52.978044 4407 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:18:52.979336 4388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:18:52.983994 4388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:18:52.985311 4388 hybrid_clock.cc:648] HybridClock initialized: now 1786277932985191 us; error 109 us; skew 500 ppm
I20260809 12:18:52.985844 4388 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:18:52 dist-test-slave-39l8 krb5kdc[3534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277932, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:18:52.991845 4388 init.cc:377] Logged in from keytab as kudu/127.19.183.193@KRBTEST.COM (short username kudu)
I20260809 12:18:52.999740 4388 webserver.cc:460] Webserver started at http://127.19.183.193:36709/ using document root <none> and password file <none>
I20260809 12:18:53.001091 4388 fs_manager.cc:359] Metadata directory not provided
I20260809 12:18:53.001339 4388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:18:53.001907 4388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:18:53 dist-test-slave-39l8 krb5kdc[3534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277932, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
I20260809 12:18:53.032207 4388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "a61dc5246f294abcada757aa6e38a05e"
format_stamp: "Formatted at 2026-08-09 12:18:53 on dist-test-slave-39l8"
server_key: "1537c58bd451eaea22603862488ba1a6"
server_key_iv: "fe53afac80734b13e2c21053793b4301"
server_key_version: "kuduclusterkey@0"
I20260809 12:18:53.033699 4388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "a61dc5246f294abcada757aa6e38a05e"
format_stamp: "Formatted at 2026-08-09 12:18:53 on dist-test-slave-39l8"
server_key: "1537c58bd451eaea22603862488ba1a6"
server_key_iv: "fe53afac80734b13e2c21053793b4301"
server_key_version: "kuduclusterkey@0"
I20260809 12:18:53.048199 4388 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.005s sys 0.005s
I20260809 12:18:53.083711 4417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:18:53.086696 4388 fs_manager.cc:725] Time spent opening block manager: real 0.013s user 0.002s sys 0.009s
I20260809 12:18:53.086994 4388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "a61dc5246f294abcada757aa6e38a05e"
format_stamp: "Formatted at 2026-08-09 12:18:53 on dist-test-slave-39l8"
server_key: "1537c58bd451eaea22603862488ba1a6"
server_key_iv: "fe53afac80734b13e2c21053793b4301"
server_key_version: "kuduclusterkey@0"
I20260809 12:18:53.087356 4388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:18:53.146944 4388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:18:53.163604 4388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:18:53.164592 4388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:18:53.182052 4388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
Aug 09 12:18:53 dist-test-slave-39l8 krb5kdc[3534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277932, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
I20260809 12:18:53.195864 4388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:18:53.196179 4388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260809 12:18:53.196463 4388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:18:53.196655 4388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:18:53.493458 4388 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:40123
I20260809 12:18:53.493639 4586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:40123 every 8 connection(s)
I20260809 12:18:53.495635 4388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:18:53.505218 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 4388
I20260809 12:18:53.505476 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:18:53.506577 20191 external_mini_cluster.cc:1428] Setting key 3f1defa1fe7bc0c0084a124862a18b8c
Aug 09 12:18:53 dist-test-slave-39l8 krb5kdc[3534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277932, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:18:53.549657 4589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:53.500283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:39295 (local address 127.19.183.254:38693)
0809 12:18:53.519042 (+ 18759us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:53.519052 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:53.524955 (+ 5903us) server_negotiation.cc:407] Connection header received
0809 12:18:53.525034 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:53.525038 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:53.525115 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:53.525293 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:18:53.528310 (+ 3017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:53.529749 (+ 1439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:53.532603 (+ 2854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:53.533310 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:53.533407 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:53.539758 (+ 6351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:18:53.539765 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:18:53.539770 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:18:53.539862 (+ 92us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:18:53.545269 (+ 5407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:53.546491 (+ 1222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:53.546496 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:53.546500 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:53.546774 (+ 274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:53.547414 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:53.547418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:53.547421 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:53.547800 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:18:53.547942 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:53.549084 (+ 1142us) server_negotiation.cc:299] Negotiation successful
0809 12:18:53.549306 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18545,"thread_start_us":18590,"threads_started":1}
I20260809 12:18:53.552181 4595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:53.501152 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:38693 (local address 127.19.183.193:39295)
0809 12:18:53.520620 (+ 19468us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:53.520681 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 12:18:53.524563 (+ 3882us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:18:53.525758 (+ 1195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:18:53.525771 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:18:53.526576 (+ 805us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:18:53.528087 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:53.528117 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:53.530039 (+ 1922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:53.530045 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:53.532418 (+ 2373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:53.532443 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:53.533966 (+ 1523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:53.533975 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:53.534356 (+ 381us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:53.535477 (+ 1121us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:18:53.535512 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:18:53.539439 (+ 3927us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:18:53.545532 (+ 6093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:18:53.545541 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:18:53.545560 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:18:53.546350 (+ 790us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:18:53.546921 (+ 571us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:18:53.546925 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:18:53.546928 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:18:53.547265 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:18:53.548047 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:18:53.548055 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:18:53.548523 (+ 468us) client_negotiation.cc:769] Sending connection context
0809 12:18:53.549888 (+ 1365us) client_negotiation.cc:240] Negotiation successful
0809 12:18:53.550488 (+ 600us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19026,"thread_start_us":18609,"threads_started":1}
I20260809 12:18:53.554567 4587 heartbeater.cc:347] Connected to a master server at 127.19.183.254:38693
I20260809 12:18:53.555104 4587 heartbeater.cc:464] Registering TS with master...
I20260809 12:18:53.556249 4587 heartbeater.cc:511] Master 127.19.183.254:38693 requested a full tablet report, sending...
I20260809 12:18:53.559693 4259 ts_manager.cc:194] Registered new tserver with Master: a61dc5246f294abcada757aa6e38a05e (127.19.183.193:40123)
I20260809 12:18:53.562433 4259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:39295
WARNING: no policy specified for kudu/127.19.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
W20260809 12:18:53.622892 20191 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.19.183.194: real 0.101s user 0.004s sys 0.037s
I20260809 12:18:53.623893 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.183.212:55665/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:38693
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:45583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:18:53 dist-test-slave-39l8 krb5kdc[3534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277932, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
W20260809 12:18:54.334547 4599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:18:54.334952 4599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:18:54.335156 4599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:18:54.344764 4599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:18:54.344995 4599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:18:54.345201 4599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:18:54.358443 4599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:45583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.19.183.212:55665/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:38693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:18:54.360075 4599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:18:54.361582 4599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:18:54.382911 4621 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:18:54.387041 4623 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:18:54.388298 4625 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:18:54.389029 4599 server_base.cc:1034] running on GCE node
I20260809 12:18:54.389976 4599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:18:54.392815 4599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:18:54.394176 4599 hybrid_clock.cc:648] HybridClock initialized: now 1786277934394029 us; error 66 us; skew 500 ppm
I20260809 12:18:54.394817 4599 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:18:54 dist-test-slave-39l8 krb5kdc[3534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277934, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:18:54.400620 4599 init.cc:377] Logged in from keytab as kudu/127.19.183.194@KRBTEST.COM (short username kudu)
I20260809 12:18:54.406008 4599 webserver.cc:460] Webserver started at http://127.19.183.194:35869/ using document root <none> and password file <none>
I20260809 12:18:54.407310 4599 fs_manager.cc:359] Metadata directory not provided
I20260809 12:18:54.407603 4599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:18:54.408161 4599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:18:54 dist-test-slave-39l8 krb5kdc[3534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277934, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
I20260809 12:18:54.432412 4599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "f887479e197a413ab3d175d00a45891c"
format_stamp: "Formatted at 2026-08-09 12:18:54 on dist-test-slave-39l8"
server_key: "e08b4855e49e0f6a0f9e669a19cb8373"
server_key_iv: "0d4581fe1642c8203a926d63fbb43c91"
server_key_version: "kuduclusterkey@0"
I20260809 12:18:54.433648 4599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "f887479e197a413ab3d175d00a45891c"
format_stamp: "Formatted at 2026-08-09 12:18:54 on dist-test-slave-39l8"
server_key: "e08b4855e49e0f6a0f9e669a19cb8373"
server_key_iv: "0d4581fe1642c8203a926d63fbb43c91"
server_key_version: "kuduclusterkey@0"
I20260809 12:18:54.442086 4599 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.005s sys 0.005s
I20260809 12:18:54.463102 4634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:18:54.464879 4599 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.002s
I20260809 12:18:54.465149 4599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "f887479e197a413ab3d175d00a45891c"
format_stamp: "Formatted at 2026-08-09 12:18:54 on dist-test-slave-39l8"
server_key: "e08b4855e49e0f6a0f9e669a19cb8373"
server_key_iv: "0d4581fe1642c8203a926d63fbb43c91"
server_key_version: "kuduclusterkey@0"
I20260809 12:18:54.465423 4599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:18:54.483577 4599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:18:54.489632 4599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:18:54.490114 4599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:18:54.491506 4599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:18:54.493494 4599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:18:54.493585 4599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:18:54.493783 4599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:18:54.493949 4599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:18:54.567203 4587 heartbeater.cc:503] Master 127.19.183.254:38693 was elected leader, sending a full tablet report...
I20260809 12:18:54.583676 4599 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:41967
I20260809 12:18:54.583834 4796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:41967 every 8 connection(s)
I20260809 12:18:54.585366 4599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:18:54.593618 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 4599
I20260809 12:18:54.593851 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:18:54.594573 20191 external_mini_cluster.cc:1428] Setting key caa1627fceb4254025b44cb033e1a959
Aug 09 12:18:54 dist-test-slave-39l8 krb5kdc[3534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277934, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:18:54.611239 4799 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:54.589924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:44543 (local address 127.19.183.254:38693)
0809 12:18:54.591171 (+ 1247us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:54.591177 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:54.592200 (+ 1023us) server_negotiation.cc:407] Connection header received
0809 12:18:54.595653 (+ 3453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:54.595659 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:54.595753 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:54.596012 (+ 259us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:18:54.598247 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:54.599397 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:54.601139 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:54.601582 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:54.601724 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:54.605174 (+ 3450us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:18:54.605180 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:18:54.605183 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:18:54.605220 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:18:54.608087 (+ 2867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:54.608811 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:54.608824 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:54.608826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:54.608954 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:54.609404 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:54.609409 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:54.609412 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:54.609694 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:18:54.609871 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:54.610585 (+ 714us) server_negotiation.cc:299] Negotiation successful
0809 12:18:54.610839 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1005,"thread_start_us":720,"threads_started":1}
I20260809 12:18:54.612226 4800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:54.590412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:38693 (local address 127.19.183.194:44543)
0809 12:18:54.591884 (+ 1472us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:54.591944 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 12:18:54.595319 (+ 3375us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:18:54.596311 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:18:54.596319 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:18:54.597164 (+ 845us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:18:54.598068 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:54.598089 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:54.599552 (+ 1463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:54.599558 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:54.600974 (+ 1416us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:54.600987 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:54.601822 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:54.601830 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:54.602056 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:54.602854 (+ 798us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:18:54.602876 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:18:54.604948 (+ 2072us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:18:54.608233 (+ 3285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:18:54.608241 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:18:54.608256 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:18:54.608670 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:18:54.609081 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:18:54.609086 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:18:54.609090 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:18:54.609283 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:18:54.609833 (+ 550us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:18:54.609842 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:18:54.610169 (+ 327us) client_negotiation.cc:769] Sending connection context
0809 12:18:54.610466 (+ 297us) client_negotiation.cc:240] Negotiation successful
0809 12:18:54.611034 (+ 568us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1209,"thread_start_us":905,"threads_started":1}
I20260809 12:18:54.614194 4797 heartbeater.cc:347] Connected to a master server at 127.19.183.254:38693
I20260809 12:18:54.614514 4797 heartbeater.cc:464] Registering TS with master...
I20260809 12:18:54.615290 4797 heartbeater.cc:511] Master 127.19.183.254:38693 requested a full tablet report, sending...
I20260809 12:18:54.616977 4259 ts_manager.cc:194] Registered new tserver with Master: f887479e197a413ab3d175d00a45891c (127.19.183.194:41967)
I20260809 12:18:54.617923 4259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:44543
WARNING: no policy specified for kudu/127.19.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
I20260809 12:18:54.677709 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.183.212:55665/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:38693
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:45583
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:18:55.249385 4803 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:18:55.249807 4803 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:18:55.250072 4803 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:18:55.259789 4803 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:18:55.259970 4803 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:18:55.260156 4803 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:18:55.272564 4803 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:45583
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.19.183.212:55665/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:38693
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:18:55.274353 4803 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:18:55.275918 4803 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:18:55.293844 4813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:18:55.293740 4816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:18:55.300859 4812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:18:55.301887 4803 server_base.cc:1034] running on GCE node
I20260809 12:18:55.302881 4803 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:18:55.303910 4803 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:18:55.305183 4803 hybrid_clock.cc:648] HybridClock initialized: now 1786277935305118 us; error 61 us; skew 500 ppm
I20260809 12:18:55.305487 4803 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:18:55 dist-test-slave-39l8 krb5kdc[3534](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277935, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:18:55.313472 4803 init.cc:377] Logged in from keytab as kudu/127.19.183.195@KRBTEST.COM (short username kudu)
I20260809 12:18:55.315574 4803 webserver.cc:460] Webserver started at http://127.19.183.195:44237/ using document root <none> and password file <none>
I20260809 12:18:55.316319 4803 fs_manager.cc:359] Metadata directory not provided
I20260809 12:18:55.316479 4803 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:18:55.316910 4803 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:18:55 dist-test-slave-39l8 krb5kdc[3534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277935, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
I20260809 12:18:55.335080 4803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "599317966eaf4d36b6484889dbf66b00"
format_stamp: "Formatted at 2026-08-09 12:18:55 on dist-test-slave-39l8"
server_key: "5ffa15cb65db963c5b5686128c9cb997"
server_key_iv: "6341b85069f9010aac1368439b3e6a30"
server_key_version: "kuduclusterkey@0"
I20260809 12:18:55.336160 4803 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "599317966eaf4d36b6484889dbf66b00"
format_stamp: "Formatted at 2026-08-09 12:18:55 on dist-test-slave-39l8"
server_key: "5ffa15cb65db963c5b5686128c9cb997"
server_key_iv: "6341b85069f9010aac1368439b3e6a30"
server_key_version: "kuduclusterkey@0"
I20260809 12:18:55.342494 4803 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.001s
I20260809 12:18:55.359299 4828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:18:55.361166 4803 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260809 12:18:55.361377 4803 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "599317966eaf4d36b6484889dbf66b00"
format_stamp: "Formatted at 2026-08-09 12:18:55 on dist-test-slave-39l8"
server_key: "5ffa15cb65db963c5b5686128c9cb997"
server_key_iv: "6341b85069f9010aac1368439b3e6a30"
server_key_version: "kuduclusterkey@0"
I20260809 12:18:55.361747 4803 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:18:55.395890 4803 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:18:55.400826 4803 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:18:55.401296 4803 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:18:55.402791 4803 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:18:55.405263 4803 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:18:55.405417 4803 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:18:55.405607 4803 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:18:55.405740 4803 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:18:55.459724 4803 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:40101
I20260809 12:18:55.459776 4996 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:40101 every 8 connection(s)
I20260809 12:18:55.461089 4803 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:18:55.463316 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 4803
I20260809 12:18:55.463546 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:18:55.464098 20191 external_mini_cluster.cc:1428] Setting key 75d03fe14ff1bc16717cac38a6b693bd
Aug 09 12:18:55 dist-test-slave-39l8 krb5kdc[3534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277935, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:18:55.487016 5000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:55.464324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:38223 (local address 127.19.183.254:38693)
0809 12:18:55.465036 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:55.465043 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:55.466252 (+ 1209us) server_negotiation.cc:407] Connection header received
0809 12:18:55.468938 (+ 2686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:55.468946 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:55.469082 (+ 136us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:55.469390 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:18:55.471209 (+ 1819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.472957 (+ 1748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.474971 (+ 2014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.475429 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.475523 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:55.479556 (+ 4033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:18:55.479565 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:18:55.479572 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:18:55.479631 (+ 59us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:18:55.483531 (+ 3900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:55.484417 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:55.484424 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:55.484427 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:55.484665 (+ 238us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:55.485304 (+ 639us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:55.485309 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:55.485312 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:55.485718 (+ 406us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:18:55.485832 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:55.486468 (+ 636us) server_negotiation.cc:299] Negotiation successful
0809 12:18:55.486633 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":570,"thread_start_us":409,"threads_started":1}
I20260809 12:18:55.487910 5001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:55.464974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:38693 (local address 127.19.183.195:38223)
0809 12:18:55.466054 (+ 1080us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:55.466107 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 12:18:55.468605 (+ 2498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:18:55.469593 (+ 988us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:18:55.469601 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:18:55.470154 (+ 553us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:18:55.471036 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:55.471056 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.473132 (+ 2076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.473137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:55.474849 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:55.474867 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.475708 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.475714 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:55.475943 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:55.476585 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:18:55.476605 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:18:55.479276 (+ 2671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:18:55.483671 (+ 4395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:18:55.483677 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:18:55.483687 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:18:55.484263 (+ 576us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:18:55.484774 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:18:55.484778 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:18:55.484780 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:18:55.485164 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:18:55.485826 (+ 662us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:18:55.485831 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:18:55.486149 (+ 318us) client_negotiation.cc:769] Sending connection context
0809 12:18:55.486381 (+ 232us) client_negotiation.cc:240] Negotiation successful
0809 12:18:55.486832 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":820,"thread_start_us":565,"threads_started":1}
I20260809 12:18:55.489602 4998 heartbeater.cc:347] Connected to a master server at 127.19.183.254:38693
I20260809 12:18:55.489995 4998 heartbeater.cc:464] Registering TS with master...
I20260809 12:18:55.491242 4998 heartbeater.cc:511] Master 127.19.183.254:38693 requested a full tablet report, sending...
I20260809 12:18:55.493487 4259 ts_manager.cc:194] Registered new tserver with Master: 599317966eaf4d36b6484889dbf66b00 (127.19.183.195:40101)
I20260809 12:18:55.494549 4259 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:38223
I20260809 12:18:55.505147 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:18:55.527582 5000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:55.510816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49722 (local address 127.19.183.254:38693)
0809 12:18:55.510956 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:55.510960 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:55.511983 (+ 1023us) server_negotiation.cc:407] Connection header received
0809 12:18:55.512259 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:55.512263 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:55.512345 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:55.512446 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:18:55.514359 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.515521 (+ 1162us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.517411 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.518226 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.518395 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:55.521862 (+ 3467us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:18:55.521867 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:18:55.521871 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:18:55.521901 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:18:55.525086 (+ 3185us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:55.525817 (+ 731us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:55.525820 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:55.525822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:55.525939 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:55.526296 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:55.526299 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:55.526301 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:55.526542 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:18:55.526769 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:55.527104 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 12:18:55.527289 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 12:18:55.531666 4259 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49722:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:18:55.542265 4259 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:18:55.589656 5017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:55.559639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35028 (local address 127.19.183.195:40101)
0809 12:18:55.570030 (+ 10391us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:55.570036 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:55.570089 (+ 53us) server_negotiation.cc:407] Connection header received
0809 12:18:55.570199 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:55.570207 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:55.570383 (+ 176us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:55.570793 (+ 410us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:18:55.575181 (+ 4388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.578464 (+ 3283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.584905 (+ 6441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.587213 (+ 2308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.587486 (+ 273us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:55.587669 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:55.588966 (+ 1297us) server_negotiation.cc:299] Negotiation successful
0809 12:18:55.589152 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10237,"thread_start_us":9920,"threads_started":1}
I20260809 12:18:55.590198 5016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:55.559107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46486 (local address 127.19.183.193:40123)
0809 12:18:55.568951 (+ 9844us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:55.568964 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:55.569002 (+ 38us) server_negotiation.cc:407] Connection header received
0809 12:18:55.569160 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:55.569173 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:55.571559 (+ 2386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:55.571921 (+ 362us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:18:55.573937 (+ 2016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.578332 (+ 4395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.585654 (+ 7322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.587923 (+ 2269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.588229 (+ 306us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:55.588470 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:55.589300 (+ 830us) server_negotiation.cc:299] Negotiation successful
0809 12:18:55.589497 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9590,"thread_start_us":9240,"threads_started":1}
I20260809 12:18:55.591295 5015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:55.558387 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:40123 (local address 127.0.0.1:46486)
0809 12:18:55.560461 (+ 2074us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:55.560511 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 12:18:55.560884 (+ 373us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:18:55.571885 (+ 11001us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:18:55.571890 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:18:55.571917 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:18:55.573679 (+ 1762us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:55.573696 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.578461 (+ 4765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.578469 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:55.585426 (+ 6957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:55.585441 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.588491 (+ 3050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.588503 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:55.588737 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:55.588770 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 12:18:55.588889 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 12:18:55.589063 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1573,"mutex_wait_us":32,"thread_start_us":3350,"threads_started":1}
I20260809 12:18:55.592635 5012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:55.558778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:40101 (local address 127.0.0.1:35028)
0809 12:18:55.562274 (+ 3496us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:55.568585 (+ 6311us) client_negotiation.cc:174] Beginning negotiation
0809 12:18:55.568810 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:18:55.571816 (+ 3006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:18:55.571826 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:18:55.571870 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:18:55.574744 (+ 2874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:55.574759 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.578823 (+ 4064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.578832 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:55.584678 (+ 5846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:55.584694 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.588362 (+ 3668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.588372 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:55.588646 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:55.588709 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 12:18:55.588839 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 12:18:55.589049 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3281,"mutex_wait_us":6,"thread_start_us":1245,"threads_started":1}
I20260809 12:18:55.597024 5018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:55.562461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:41967 (local address 127.0.0.1:41264)
0809 12:18:55.580100 (+ 17639us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:55.580125 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0809 12:18:55.580337 (+ 212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:18:55.581415 (+ 1078us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:18:55.581421 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:18:55.581464 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:18:55.583734 (+ 2270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:55.583748 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.586942 (+ 3194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.586948 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:55.593246 (+ 6298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:55.593262 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.595982 (+ 2720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.595991 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:55.596273 (+ 282us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:55.596306 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 12:18:55.596398 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 12:18:55.596517 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17389,"thread_start_us":16946,"threads_started":1}
I20260809 12:18:55.597337 5020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:55.562751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41264 (local address 127.19.183.194:41967)
0809 12:18:55.573399 (+ 10648us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:55.573406 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:55.580321 (+ 6915us) server_negotiation.cc:407] Connection header received
0809 12:18:55.580544 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:55.580553 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:55.580961 (+ 408us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:55.581416 (+ 455us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:18:55.584020 (+ 2604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.586618 (+ 2598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.593428 (+ 6810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.595558 (+ 2130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.595904 (+ 346us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:55.596113 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:55.596483 (+ 370us) server_negotiation.cc:299] Negotiation successful
0809 12:18:55.596680 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10422,"thread_start_us":11776,"threads_started":1}
I20260809 12:18:55.597332 4905 tablet_service.cc:1467] Processing CreateTablet for tablet 3f98a33e354942eb9dde8bd7b51577da (DEFAULT_TABLE table=test-table [id=c8f14e46ae75488f857c0bf456e84cbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:18:55.596877 4492 tablet_service.cc:1467] Processing CreateTablet for tablet 3f98a33e354942eb9dde8bd7b51577da (DEFAULT_TABLE table=test-table [id=c8f14e46ae75488f857c0bf456e84cbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:18:55.599229 4492 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f98a33e354942eb9dde8bd7b51577da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:18:55.599309 4905 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f98a33e354942eb9dde8bd7b51577da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:18:55.602418 4703 tablet_service.cc:1467] Processing CreateTablet for tablet 3f98a33e354942eb9dde8bd7b51577da (DEFAULT_TABLE table=test-table [id=c8f14e46ae75488f857c0bf456e84cbb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:18:55.604532 4703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f98a33e354942eb9dde8bd7b51577da. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:18:55.616046 5024 tablet_bootstrap.cc:492] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e: Bootstrap starting.
I20260809 12:18:55.620766 4797 heartbeater.cc:503] Master 127.19.183.254:38693 was elected leader, sending a full tablet report...
I20260809 12:18:55.621850 5024 tablet_bootstrap.cc:654] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e: Neither blocks nor log segments found. Creating new log.
I20260809 12:18:55.624728 5024 log.cc:824] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e: Log is configured to *not* fsync() on all Append() calls
I20260809 12:18:55.628870 5028 tablet_bootstrap.cc:492] T 3f98a33e354942eb9dde8bd7b51577da P f887479e197a413ab3d175d00a45891c: Bootstrap starting.
I20260809 12:18:55.629575 5024 tablet_bootstrap.cc:492] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e: No bootstrap required, opened a new log
I20260809 12:18:55.629861 5024 ts_tablet_manager.cc:1397] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e: Time spent bootstrapping tablet: real 0.014s user 0.000s sys 0.012s
I20260809 12:18:55.630990 5025 tablet_bootstrap.cc:492] T 3f98a33e354942eb9dde8bd7b51577da P 599317966eaf4d36b6484889dbf66b00: Bootstrap starting.
I20260809 12:18:55.633965 5028 tablet_bootstrap.cc:654] T 3f98a33e354942eb9dde8bd7b51577da P f887479e197a413ab3d175d00a45891c: Neither blocks nor log segments found. Creating new log.
I20260809 12:18:55.635816 5028 log.cc:824] T 3f98a33e354942eb9dde8bd7b51577da P f887479e197a413ab3d175d00a45891c: Log is configured to *not* fsync() on all Append() calls
I20260809 12:18:55.637362 5025 tablet_bootstrap.cc:654] T 3f98a33e354942eb9dde8bd7b51577da P 599317966eaf4d36b6484889dbf66b00: Neither blocks nor log segments found. Creating new log.
I20260809 12:18:55.639585 5025 log.cc:824] T 3f98a33e354942eb9dde8bd7b51577da P 599317966eaf4d36b6484889dbf66b00: Log is configured to *not* fsync() on all Append() calls
I20260809 12:18:55.644358 5024 raft_consensus.cc:348] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a61dc5246f294abcada757aa6e38a05e" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40123 } } peers { permanent_uuid: "599317966eaf4d36b6484889dbf66b00" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 40101 } } peers { permanent_uuid: "f887479e197a413ab3d175d00a45891c" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41967 } }
I20260809 12:18:55.644624 5025 tablet_bootstrap.cc:492] T 3f98a33e354942eb9dde8bd7b51577da P 599317966eaf4d36b6484889dbf66b00: No bootstrap required, opened a new log
I20260809 12:18:55.644624 5028 tablet_bootstrap.cc:492] T 3f98a33e354942eb9dde8bd7b51577da P f887479e197a413ab3d175d00a45891c: No bootstrap required, opened a new log
I20260809 12:18:55.644886 5024 raft_consensus.cc:374] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:18:55.645049 5028 ts_tablet_manager.cc:1397] T 3f98a33e354942eb9dde8bd7b51577da P f887479e197a413ab3d175d00a45891c: Time spent bootstrapping tablet: real 0.016s user 0.000s sys 0.009s
I20260809 12:18:55.645098 5025 ts_tablet_manager.cc:1397] T 3f98a33e354942eb9dde8bd7b51577da P 599317966eaf4d36b6484889dbf66b00: Time spent bootstrapping tablet: real 0.018s user 0.003s sys 0.008s
I20260809 12:18:55.645093 5024 raft_consensus.cc:729] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a61dc5246f294abcada757aa6e38a05e, State: Initialized, Role: FOLLOWER
I20260809 12:18:55.645783 5024 consensus_queue.cc:260] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a61dc5246f294abcada757aa6e38a05e" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40123 } } peers { permanent_uuid: "599317966eaf4d36b6484889dbf66b00" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 40101 } } peers { permanent_uuid: "f887479e197a413ab3d175d00a45891c" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41967 } }
I20260809 12:18:55.648469 5024 ts_tablet_manager.cc:1428] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e: Time spent starting tablet: real 0.018s user 0.008s sys 0.010s
I20260809 12:18:55.662258 5028 raft_consensus.cc:348] T 3f98a33e354942eb9dde8bd7b51577da P f887479e197a413ab3d175d00a45891c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a61dc5246f294abcada757aa6e38a05e" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40123 } } peers { permanent_uuid: "599317966eaf4d36b6484889dbf66b00" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 40101 } } peers { permanent_uuid: "f887479e197a413ab3d175d00a45891c" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41967 } }
I20260809 12:18:55.662465 5025 raft_consensus.cc:348] T 3f98a33e354942eb9dde8bd7b51577da P 599317966eaf4d36b6484889dbf66b00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a61dc5246f294abcada757aa6e38a05e" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40123 } } peers { permanent_uuid: "599317966eaf4d36b6484889dbf66b00" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 40101 } } peers { permanent_uuid: "f887479e197a413ab3d175d00a45891c" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41967 } }
I20260809 12:18:55.662998 5028 raft_consensus.cc:374] T 3f98a33e354942eb9dde8bd7b51577da P f887479e197a413ab3d175d00a45891c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:18:55.663105 5025 raft_consensus.cc:374] T 3f98a33e354942eb9dde8bd7b51577da P 599317966eaf4d36b6484889dbf66b00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:18:55.663228 5025 raft_consensus.cc:729] T 3f98a33e354942eb9dde8bd7b51577da P 599317966eaf4d36b6484889dbf66b00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 599317966eaf4d36b6484889dbf66b00, State: Initialized, Role: FOLLOWER
I20260809 12:18:55.663218 5028 raft_consensus.cc:729] T 3f98a33e354942eb9dde8bd7b51577da P f887479e197a413ab3d175d00a45891c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f887479e197a413ab3d175d00a45891c, State: Initialized, Role: FOLLOWER
I20260809 12:18:55.664043 5028 consensus_queue.cc:260] T 3f98a33e354942eb9dde8bd7b51577da P f887479e197a413ab3d175d00a45891c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a61dc5246f294abcada757aa6e38a05e" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40123 } } peers { permanent_uuid: "599317966eaf4d36b6484889dbf66b00" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 40101 } } peers { permanent_uuid: "f887479e197a413ab3d175d00a45891c" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41967 } }
I20260809 12:18:55.664043 5025 consensus_queue.cc:260] T 3f98a33e354942eb9dde8bd7b51577da P 599317966eaf4d36b6484889dbf66b00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a61dc5246f294abcada757aa6e38a05e" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40123 } } peers { permanent_uuid: "599317966eaf4d36b6484889dbf66b00" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 40101 } } peers { permanent_uuid: "f887479e197a413ab3d175d00a45891c" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41967 } }
I20260809 12:18:55.666122 4998 heartbeater.cc:503] Master 127.19.183.254:38693 was elected leader, sending a full tablet report...
I20260809 12:18:55.666857 5025 ts_tablet_manager.cc:1428] T 3f98a33e354942eb9dde8bd7b51577da P 599317966eaf4d36b6484889dbf66b00: Time spent starting tablet: real 0.022s user 0.004s sys 0.020s
I20260809 12:18:55.666887 5028 ts_tablet_manager.cc:1428] T 3f98a33e354942eb9dde8bd7b51577da P f887479e197a413ab3d175d00a45891c: Time spent starting tablet: real 0.022s user 0.012s sys 0.011s
W20260809 12:18:55.713279 4999 tablet.cc:2369] T 3f98a33e354942eb9dde8bd7b51577da P 599317966eaf4d36b6484889dbf66b00: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:18:55.750823 4588 tablet.cc:2369] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:18:55.839514 4798 tablet.cc:2369] T 3f98a33e354942eb9dde8bd7b51577da P f887479e197a413ab3d175d00a45891c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:18:55.851482 5033 raft_consensus.cc:482] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:18:55.851960 5033 raft_consensus.cc:504] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a61dc5246f294abcada757aa6e38a05e" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40123 } } peers { permanent_uuid: "599317966eaf4d36b6484889dbf66b00" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 40101 } } peers { permanent_uuid: "f887479e197a413ab3d175d00a45891c" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41967 } }
I20260809 12:18:55.854592 5033 leader_election.cc:288] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 599317966eaf4d36b6484889dbf66b00 (127.19.183.195:40101), f887479e197a413ab3d175d00a45891c (127.19.183.194:41967)
I20260809 12:18:55.864845 5020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:55.854975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:33507 (local address 127.19.183.194:41967)
0809 12:18:55.855257 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:55.855263 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:55.857002 (+ 1739us) server_negotiation.cc:407] Connection header received
0809 12:18:55.857265 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:55.857270 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:55.857341 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:55.857431 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:18:55.858773 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.859741 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.862688 (+ 2947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.863535 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.863765 (+ 230us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:55.863878 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:55.864284 (+ 406us) server_negotiation.cc:299] Negotiation successful
0809 12:18:55.864422 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:18:55.865130 5036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:55.854824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:41967 (local address 127.19.183.193:33507)
0809 12:18:55.856854 (+ 2030us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:55.856870 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:18:55.857111 (+ 241us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:18:55.857453 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:18:55.857458 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:18:55.857485 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:18:55.858590 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:55.858605 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.859876 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.859882 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:55.862508 (+ 2626us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:55.862522 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.863920 (+ 1398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.863933 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:55.864102 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:55.864127 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 12:18:55.864245 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 12:18:55.864386 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1882,"thread_start_us":1495,"threads_started":1}
I20260809 12:18:55.865759 5017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:55.855031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:54793 (local address 127.19.183.195:40101)
0809 12:18:55.855256 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:55.855263 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:55.857779 (+ 2516us) server_negotiation.cc:407] Connection header received
0809 12:18:55.858067 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:55.858076 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:55.858136 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:55.858229 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:18:55.859500 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.860258 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.863225 (+ 2967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.864433 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.864655 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:55.864756 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:55.865247 (+ 491us) server_negotiation.cc:299] Negotiation successful
0809 12:18:55.865368 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:18:55.866055 5038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:55.855936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:40101 (local address 127.19.183.193:54793)
0809 12:18:55.857611 (+ 1675us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:55.857634 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:18:55.857913 (+ 279us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:18:55.858255 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:18:55.858260 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:18:55.858283 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:18:55.859298 (+ 1015us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:55.859310 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.860458 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.860462 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:55.863035 (+ 2573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:18:55.863047 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.864808 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.864815 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:18:55.865023 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:55.865071 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 12:18:55.865203 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 12:18:55.865330 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1490,"thread_start_us":1144,"threads_started":1}
I20260809 12:18:55.866833 4735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f98a33e354942eb9dde8bd7b51577da" candidate_uuid: "a61dc5246f294abcada757aa6e38a05e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f887479e197a413ab3d175d00a45891c" is_pre_election: true
I20260809 12:18:55.867694 4934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f98a33e354942eb9dde8bd7b51577da" candidate_uuid: "a61dc5246f294abcada757aa6e38a05e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "599317966eaf4d36b6484889dbf66b00" is_pre_election: true
I20260809 12:18:55.867928 4735 raft_consensus.cc:2454] T 3f98a33e354942eb9dde8bd7b51577da P f887479e197a413ab3d175d00a45891c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a61dc5246f294abcada757aa6e38a05e in term 0.
I20260809 12:18:55.868605 4934 raft_consensus.cc:2454] T 3f98a33e354942eb9dde8bd7b51577da P 599317966eaf4d36b6484889dbf66b00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a61dc5246f294abcada757aa6e38a05e in term 0.
I20260809 12:18:55.869212 4420 leader_election.cc:302] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a61dc5246f294abcada757aa6e38a05e, f887479e197a413ab3d175d00a45891c; no voters:
I20260809 12:18:55.869740 5033 raft_consensus.cc:2781] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:18:55.869992 5033 raft_consensus.cc:482] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:18:55.870249 5033 raft_consensus.cc:3037] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:18:55.874061 5033 raft_consensus.cc:504] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a61dc5246f294abcada757aa6e38a05e" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40123 } } peers { permanent_uuid: "599317966eaf4d36b6484889dbf66b00" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 40101 } } peers { permanent_uuid: "f887479e197a413ab3d175d00a45891c" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41967 } }
I20260809 12:18:55.875124 5033 leader_election.cc:288] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e [CANDIDATE]: Term 1 election: Requested vote from peers 599317966eaf4d36b6484889dbf66b00 (127.19.183.195:40101), f887479e197a413ab3d175d00a45891c (127.19.183.194:41967)
I20260809 12:18:55.875574 4934 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f98a33e354942eb9dde8bd7b51577da" candidate_uuid: "a61dc5246f294abcada757aa6e38a05e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "599317966eaf4d36b6484889dbf66b00"
I20260809 12:18:55.875742 4735 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f98a33e354942eb9dde8bd7b51577da" candidate_uuid: "a61dc5246f294abcada757aa6e38a05e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f887479e197a413ab3d175d00a45891c"
I20260809 12:18:55.875810 4934 raft_consensus.cc:3037] T 3f98a33e354942eb9dde8bd7b51577da P 599317966eaf4d36b6484889dbf66b00 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:18:55.876016 4735 raft_consensus.cc:3037] T 3f98a33e354942eb9dde8bd7b51577da P f887479e197a413ab3d175d00a45891c [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:18:55.879319 4934 raft_consensus.cc:2454] T 3f98a33e354942eb9dde8bd7b51577da P 599317966eaf4d36b6484889dbf66b00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a61dc5246f294abcada757aa6e38a05e in term 1.
I20260809 12:18:55.879791 4735 raft_consensus.cc:2454] T 3f98a33e354942eb9dde8bd7b51577da P f887479e197a413ab3d175d00a45891c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a61dc5246f294abcada757aa6e38a05e in term 1.
I20260809 12:18:55.879806 4424 leader_election.cc:302] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 599317966eaf4d36b6484889dbf66b00, a61dc5246f294abcada757aa6e38a05e; no voters:
I20260809 12:18:55.880388 5033 raft_consensus.cc:2781] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:18:55.880986 5033 raft_consensus.cc:686] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e [term 1 LEADER]: Becoming Leader. State: Replica: a61dc5246f294abcada757aa6e38a05e, State: Running, Role: LEADER
I20260809 12:18:55.881672 5033 consensus_queue.cc:237] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a61dc5246f294abcada757aa6e38a05e" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40123 } } peers { permanent_uuid: "599317966eaf4d36b6484889dbf66b00" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 40101 } } peers { permanent_uuid: "f887479e197a413ab3d175d00a45891c" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41967 } }
I20260809 12:18:55.887102 4259 catalog_manager.cc:5495] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e reported cstate change: term changed from 0 to 1, leader changed from <none> to a61dc5246f294abcada757aa6e38a05e (127.19.183.193). New cstate: current_term: 1 leader_uuid: "a61dc5246f294abcada757aa6e38a05e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a61dc5246f294abcada757aa6e38a05e" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "599317966eaf4d36b6484889dbf66b00" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 40101 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f887479e197a413ab3d175d00a45891c" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41967 } health_report { overall_health: UNKNOWN } } }
I20260809 12:18:55.940131 5016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:55.933016 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46494 (local address 127.19.183.193:40123)
0809 12:18:55.933252 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:55.933257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:55.933275 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:18:55.933366 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:55.933370 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:55.933450 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:55.933606 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:18:55.934095 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.935325 (+ 1230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.937655 (+ 2330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:55.938230 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:55.938326 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:55.939038 (+ 712us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:18:55.939596 (+ 558us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:18:55.939734 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:55.939818 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 12:18:55.939884 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 12:18:55.951027 4735 raft_consensus.cc:1261] T 3f98a33e354942eb9dde8bd7b51577da P f887479e197a413ab3d175d00a45891c [term 1 FOLLOWER]: Refusing update from remote peer a61dc5246f294abcada757aa6e38a05e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:18:55.951197 4934 raft_consensus.cc:1261] T 3f98a33e354942eb9dde8bd7b51577da P 599317966eaf4d36b6484889dbf66b00 [term 1 FOLLOWER]: Refusing update from remote peer a61dc5246f294abcada757aa6e38a05e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:18:55.952203 5041 consensus_queue.cc:1035] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e [LEADER]: Connected to new peer: Peer: permanent_uuid: "f887479e197a413ab3d175d00a45891c" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.001s
I20260809 12:18:55.952870 5033 consensus_queue.cc:1035] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e [LEADER]: Connected to new peer: Peer: permanent_uuid: "599317966eaf4d36b6484889dbf66b00" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 40101 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:18:55.989297 4259 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:18:55.989646 4259 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49722:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:18:55.994185 4259 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0773e4c8c8ea4d3583615cb85ac297dd: Sending DeleteTablet for 3 replicas of tablet 3f98a33e354942eb9dde8bd7b51577da
I20260809 12:18:55.995653 4905 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f98a33e354942eb9dde8bd7b51577da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:18:55 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:35028
I20260809 12:18:55.995913 4703 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f98a33e354942eb9dde8bd7b51577da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:18:55 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:41264
I20260809 12:18:55.996778 4492 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f98a33e354942eb9dde8bd7b51577da with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:18:55 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:46486
I20260809 12:18:55.999285 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 4388
I20260809 12:18:56.009219 5063 tablet_replica.cc:330] T 3f98a33e354942eb9dde8bd7b51577da P f887479e197a413ab3d175d00a45891c: stopping tablet replica
I20260809 12:18:56.009971 5063 raft_consensus.cc:2229] T 3f98a33e354942eb9dde8bd7b51577da P f887479e197a413ab3d175d00a45891c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:18:56.011034 5063 raft_consensus.cc:2258] T 3f98a33e354942eb9dde8bd7b51577da P f887479e197a413ab3d175d00a45891c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:18:56.020308 5062 tablet_replica.cc:330] T 3f98a33e354942eb9dde8bd7b51577da P 599317966eaf4d36b6484889dbf66b00: stopping tablet replica
I20260809 12:18:56.020953 5062 raft_consensus.cc:2229] T 3f98a33e354942eb9dde8bd7b51577da P 599317966eaf4d36b6484889dbf66b00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:18:56.021983 5062 raft_consensus.cc:2258] T 3f98a33e354942eb9dde8bd7b51577da P 599317966eaf4d36b6484889dbf66b00 [term 1 FOLLOWER]: Raft consensus is shut down!
Aug 09 12:18:56 dist-test-slave-39l8 krb5kdc[3534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.193@KRBTEST.COM
I20260809 12:18:56.025377 5062 ts_tablet_manager.cc:1905] T 3f98a33e354942eb9dde8bd7b51577da P 599317966eaf4d36b6484889dbf66b00: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:18:56.027143 5063 ts_tablet_manager.cc:1905] T 3f98a33e354942eb9dde8bd7b51577da P f887479e197a413ab3d175d00a45891c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:18:56.032912 5065 tablet_replica.cc:330] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e: stopping tablet replica
I20260809 12:18:56.033802 5065 raft_consensus.cc:2229] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:18:56.035442 5065 raft_consensus.cc:2258] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:18:56.037552 5063 ts_tablet_manager.cc:1918] T 3f98a33e354942eb9dde8bd7b51577da P f887479e197a413ab3d175d00a45891c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:18:56.037932 5063 log.cc:1196] T 3f98a33e354942eb9dde8bd7b51577da P f887479e197a413ab3d175d00a45891c: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/3f98a33e354942eb9dde8bd7b51577da
I20260809 12:18:56.038255 5065 ts_tablet_manager.cc:1905] T 3f98a33e354942eb9dde8bd7b51577da P a61dc5246f294abcada757aa6e38a05e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:18:56.038681 5016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:56.002635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46506 (local address 127.19.183.193:40123)
0809 12:18:56.002840 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:56.002847 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:56.007205 (+ 4358us) server_negotiation.cc:407] Connection header received
0809 12:18:56.007417 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:56.007422 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:56.007499 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:56.007620 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:18:56.009710 (+ 2090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:56.010950 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:56.017438 (+ 6488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:56.018119 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:56.018261 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:56.025453 (+ 7192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:18:56.025469 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:18:56.025487 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:18:56.025561 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:18:56.032187 (+ 6626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:56.033339 (+ 1152us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:56.033347 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:56.033350 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:56.033786 (+ 436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:56.034525 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:56.034530 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:56.034534 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:56.035180 (+ 646us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:18:56.037309 (+ 2129us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:56.037534 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 12:18:56.037869 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 12:18:56.039253 5063 ts_tablet_manager.cc:1939] T 3f98a33e354942eb9dde8bd7b51577da P f887479e197a413ab3d175d00a45891c: Deleting consensus metadata
I20260809 12:18:56.039803 4580 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:18:56.041862 4235 catalog_manager.cc:4841] TS f887479e197a413ab3d175d00a45891c (127.19.183.194:41967): tablet 3f98a33e354942eb9dde8bd7b51577da (table test-table [id=c8f14e46ae75488f857c0bf456e84cbb]) successfully deleted
I20260809 12:18:56.046213 5062 ts_tablet_manager.cc:1918] T 3f98a33e354942eb9dde8bd7b51577da P 599317966eaf4d36b6484889dbf66b00: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:18:56.046360 5062 log.cc:1196] T 3f98a33e354942eb9dde8bd7b51577da P 599317966eaf4d36b6484889dbf66b00: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/3f98a33e354942eb9dde8bd7b51577da
I20260809 12:18:56.046838 5062 ts_tablet_manager.cc:1939] T 3f98a33e354942eb9dde8bd7b51577da P 599317966eaf4d36b6484889dbf66b00: Deleting consensus metadata
I20260809 12:18:56.048611 4236 catalog_manager.cc:4841] TS 599317966eaf4d36b6484889dbf66b00 (127.19.183.195:40101): tablet 3f98a33e354942eb9dde8bd7b51577da (table test-table [id=c8f14e46ae75488f857c0bf456e84cbb]) successfully deleted
I20260809 12:18:56.144208 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 4388
I20260809 12:18:56.173398 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 4599
W20260809 12:18:56.173417 4235 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.183.193:40123 (error 108)
I20260809 12:18:56.175712 5069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:56.174957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:40123 (local address 127.0.0.1:46512)
0809 12:18:56.175415 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:56.175534 (+ 119us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:40123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":249,"threads_started":1}
W20260809 12:18:56.176051 4235 catalog_manager.cc:4584] TS a61dc5246f294abcada757aa6e38a05e (127.19.183.193:40123): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 3f98a33e354942eb9dde8bd7b51577da: Network error: Client connection negotiation failed: client connection to 127.19.183.193:40123: connect: Connection refused (error 111)
Aug 09 12:18:56 dist-test-slave-39l8 krb5kdc[3534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.194@KRBTEST.COM
I20260809 12:18:56.190363 5020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:56.174306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41280 (local address 127.19.183.194:41967)
0809 12:18:56.174508 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:56.174514 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:56.174537 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:18:56.174680 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:56.174685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:56.174800 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:56.174909 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:18:56.176419 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:56.177129 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:56.178930 (+ 1801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:56.179467 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:56.179617 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:56.184721 (+ 5104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:18:56.184734 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:18:56.184749 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:18:56.184786 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:18:56.187828 (+ 3042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:56.188434 (+ 606us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:56.188442 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:56.188444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:56.188585 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:56.188908 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:56.188911 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:56.188913 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:56.189288 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:18:56.189508 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:56.189808 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 12:18:56.189989 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:18:56.191165 4790 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:18:56.229676 5069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:56.229127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:40123 (local address 127.0.0.1:46526)
0809 12:18:56.229366 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:56.229488 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:40123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 12:18:56.230793 5069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:56.230353 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:40123 (local address 127.0.0.1:46536)
0809 12:18:56.230517 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:56.230602 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:40123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 12:18:56.280357 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 4599
I20260809 12:18:56.291518 5069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:56.290938 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:40123 (local address 127.0.0.1:46544)
0809 12:18:56.291183 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:56.291305 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:40123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 12:18:56.292689 5069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:56.292232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:40123 (local address 127.0.0.1:46560)
0809 12:18:56.292408 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:56.292505 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:40123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 12:18:56.306206 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 4803
Aug 09 12:18:56 dist-test-slave-39l8 krb5kdc[3534](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.195@KRBTEST.COM
I20260809 12:18:56.321105 5017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:56.307182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35032 (local address 127.19.183.195:40101)
0809 12:18:56.307377 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 12:18:56.307382 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:18:56.307404 (+ 22us) server_negotiation.cc:407] Connection header received
0809 12:18:56.307473 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:18:56.307476 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:18:56.307536 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:18:56.307658 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:18:56.308948 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:56.309676 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:56.311048 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:18:56.311456 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:18:56.311591 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:18:56.315802 (+ 4211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:18:56.315816 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:18:56.315832 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:18:56.315867 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:18:56.318550 (+ 2683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:56.319312 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:56.319320 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:56.319322 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:56.319450 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:18:56.319780 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:18:56.319783 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:18:56.319785 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:18:56.320147 (+ 362us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:18:56.320381 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0809 12:18:56.320649 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 12:18:56.320792 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 12:18:56.321826 4991 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:18:56.373277 5069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:56.372788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:40123 (local address 127.0.0.1:46566)
0809 12:18:56.373020 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:56.373107 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:40123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 12:18:56.374330 5069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:56.373924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:40123 (local address 127.0.0.1:46576)
0809 12:18:56.374120 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:56.374196 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:40123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260809 12:18:56.411715 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 4803
I20260809 12:18:56.443217 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 4216
I20260809 12:18:56.444437 4308 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:18:56.558064 5069 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:18:56.557271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:40123 (local address 127.0.0.1:46592)
0809 12:18:56.557557 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 12:18:56.557721 (+ 164us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:40123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
I20260809 12:18:56.558545 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 4216
I20260809 12:18:56.591749 20191 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 12:18:56.937233 20191 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 12:18:56.937500 20191 mini_ranger.cc:66] Stopping Ranger...
I20260809 12:18:57.292672 20191 mini_ranger.cc:68] Stopped Ranger
2026-08-09 12:18:57.293 UTC [3580] LOG: received smart shutdown request
2026-08-09 12:18:57.296 UTC [3580] LOG: background worker "logical replication launcher" (PID 3595) exited with exit code 1
2026-08-09 12:18:57.297 UTC [3588] LOG: shutting down
2026-08-09 12:18:57.442 UTC [3580] LOG: database system is shut down
2026-08-09T12:18:57Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (53842 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:18:57 dist-test-slave-39l8 krb5kdc[5090](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:18:57 dist-test-slave-39l8 krb5kdc[5090](info): set up 2 sockets
Aug 09 12:18:57 dist-test-slave-39l8 krb5kdc[5090](info): commencing operation
krb5kdc: starting...
W20260809 12:18:58.881660 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.004s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:18:59 dist-test-slave-39l8 krb5kdc[5090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:18:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:18:59Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.19.183.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.19.183.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.19.183.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/rangeradmin_127.19.183.212@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.19.183.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/rangeradmin_127.19.183.212@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.19.183.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.19.183.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.19.183.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/rangerlookup_127.19.183.212@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.19.183.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/rangerlookup_127.19.183.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.183.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.19.183.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.183.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/HTTP_127.19.183.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.183.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/HTTP_127.19.183.212@KRBTEST.COM.keytab.
I20260809 12:18:59.253212 20191 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskqUnAOm/build/asan/bin/postgres/pg_ctl -D /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/postgres -l logfile start
2026-08-09 12:19:01.531 UTC [5137] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 12:19:01.531 UTC [5137] LOG: listening on IPv4 address "127.19.183.212", port 42039
2026-08-09 12:19:01.534 UTC [5137] LOG: listening on Unix socket "/tmp/.s.PGSQL.42039"
2026-08-09 12:19:01.558 UTC [5141] LOG: database system was shut down at 2026-08-09 12:19:00 UTC
2026-08-09 12:19:01.561 UTC [5137] LOG: database system is ready to accept connections
I20260809 12:19:02.705168 20191 mini_postgres.cc:96] Postgres bound to 42039
2026-08-09 12:19:02.729 UTC [5158] FATAL: database "slave" does not exist
127.19.183.212:42039 - accepting connections
I20260809 12:19:02.730504 20191 mini_ranger.cc:161] Starting Ranger...
I20260809 12:19:02.762403 20191 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 12:19:03.183513 20191 mini_ranger.cc:87] Created ranger Postgres database
I20260809 12:19:03.183656 20191 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ranger-admin
2026-08-09 12:19:03,475 [I] DB FLAVOR :POSTGRES
2026-08-09 12:19:03,476 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 12:19:03,476 [I] Checking connection..
2026-08-09 12:19:03,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:42039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 12:19:03,729 [I] Checking connection passed.
2026-08-09 12:19:03,729 [I] --------- Verifying version history table ---------
2026-08-09 12:19:03,730 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:42039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 12:19:03,993 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 12:19:03,993 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:42039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 12:19:04,251 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 12:19:04,251 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 12:19:04,252 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:42039/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 12:19:04.515 UTC [5251] WARNING: there is no transaction in progress
2026-08-09 12:19:04,524 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 12:19:04,525 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:42039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 12:19:04,778 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 12:19:04,779 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 12:19:04,779 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:42039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 12:19:05,035 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:42039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 12:19:05,287 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:42039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-39l8.c.gcp-upstream.internal','N') ;"
2026-08-09 12:19:05,526 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:42039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 12:19:05,794 [I] Table x_portal_user does not exist in database ranger
2026-08-09 12:19:05,794 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:42039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 12:19:06,030 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 12:19:06,031 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:42039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 12:19:06,286 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:42039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 12:19:06,517 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 12:19:06,517 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:42039/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 12:19:07.454 UTC [5409] WARNING: there is no transaction in progress
2026-08-09 12:19:07.469 UTC [5409] WARNING: there is no transaction in progress
2026-08-09 12:19:07.485 UTC [5409] WARNING: there is no transaction in progress
2026-08-09 12:19:07.498 UTC [5409] WARNING: there is no transaction in progress
2026-08-09 12:19:07.514 UTC [5409] WARNING: there is no transaction in progress
2026-08-09 12:19:07.531 UTC [5409] WARNING: there is no transaction in progress
2026-08-09 12:19:07.592 UTC [5409] WARNING: there is no transaction in progress
2026-08-09 12:19:07.599 UTC [5409] WARNING: there is no transaction in progress
2026-08-09 12:19:07.613 UTC [5409] WARNING: there is no transaction in progress
2026-08-09 12:19:07.623 UTC [5409] WARNING: there is no transaction in progress
2026-08-09 12:19:07.634 UTC [5409] WARNING: there is no transaction in progress
2026-08-09 12:19:07.647 UTC [5409] WARNING: there is no transaction in progress
2026-08-09 12:19:07.656 UTC [5409] WARNING: there is no transaction in progress
2026-08-09 12:19:07.662 UTC [5409] WARNING: there is no transaction in progress
2026-08-09 12:19:07.668 UTC [5409] WARNING: there is no transaction in progress
2026-08-09 12:19:07.675 UTC [5409] WARNING: there is no transaction in progress
2026-08-09 12:19:08,249 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 12:19:08,249 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:42039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 12:19:08,509 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 12:19:08,510 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:42039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 12:19:08,777 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 12:19:08,777 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:42039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 12:19:09,024 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 12:19:09,025 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:42039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 12:19:09,261 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:42039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 12:19:09,524 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:42039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-39l8.c.gcp-upstream.internal';"
2026-08-09 12:19:09,746 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 12:19:09,747 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:42039/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 12:19:09,975 [I] DB_PATCHES have already been applied
I20260809 12:19:09.981626 20191 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ranger-admin:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskqUnAOm/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 12:19:09.981716 20191 mini_ranger.cc:193] Using host: 127.19.183.212
I20260809 12:19:10.003489 20191 mini_ranger.cc:227] Ranger admin URL: http://127.19.183.212:38711
Aug 09, 2026 12:19:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:19:10 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:19:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 12:19:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 12:19:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 12:19:10 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 12:19:10 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 12:19:11.344400 20191 mini_ranger.cc:160] Time spent starting Ranger: real 8.614s user 0.005s sys 0.057s
Aug 09, 2026 12:19:31 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 12:19:31 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 12:19:31 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 12:19:31 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 12:19:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 12:19:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 12:19:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 12:19:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 12:19:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 12:19:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 12:19:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 12:19:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 12:19:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 12:19:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 12:19:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 12:19:31 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 12:19:31 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 12:19:32 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 12:19:33.160899 20191 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.19.183.212@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.19.183.212@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.19.183.212@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/rangerkms_127.19.183.212@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.19.183.212@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/rangerkms_127.19.183.212@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.19.183.212@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.19.183.212@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.19.183.212@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/HTTP_127.19.183.212@KRBTEST.COM.keytab.
Entry for principal HTTP/127.19.183.212@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/HTTP_127.19.183.212@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 12:19:33 dist-test-slave-39l8 krb5kdc[5090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277973, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 12:19:33.361656 20191 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 12:19:33.386973 20191 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 12:19:34.211663 20191 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 12:19:34.211790 20191 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ranger-kms
I20260809 12:19:34.211838 20191 mini_ranger_kms.cc:227] Using postgres at 127.19.183.212:42039
2026-08-09 12:19:34,575 [I] DB FLAVOR :POSTGRES
2026-08-09 12:19:34,575 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 12:19:34,575 [I] Checking connection
2026-08-09 12:19:34,575 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:42039/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 12:19:34,846 [I] connection success
2026-08-09 12:19:34,847 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 12:19:34,847 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 12:19:34,847 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:42039/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 12:19:35,080 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 12:19:35,080 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 12:19:35,081 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 12:19:35,081 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.19.183.212:42039/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 12:19:35,351 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 12:19:35.827191 20191 mini_ranger_kms.cc:310] Created kms service
I20260809 12:19:36.000207 20191 mini_ranger_kms.cc:326] Created kudu user
I20260809 12:19:36.057431 20191 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 12:19:36 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277973, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
I20260809 12:19:36.464957 20191 mini_ranger_kms.cc:384] Added ranger policy
I20260809 12:19:36.465271 20191 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ranger-kms:/tmp/dist-test-taskqUnAOm/build/asan/bin/postgresql.jar:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskqUnAOm/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 12:19:36.465323 20191 mini_ranger_kms.cc:240] Using host: 127.19.183.212
I20260809 12:19:36.493631 20191 mini_ranger_kms.cc:276] Ranger KMS PID: 5674
I20260809 12:19:36.494014 20191 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.19.183.212:59119
Aug 09, 2026 12:19:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 12:19:37 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 12:19:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 12:19:37 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 12:19:38 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 12:19:38 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 12:19:38 dist-test-slave-39l8 krb5kdc[5090](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786277978, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.183.212@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:19:39.272837 20191 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.911s user 0.056s sys 0.345s
I20260809 12:19:39.273092 20191 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 12:19:39.273280 20191 mini_ranger_kms.cc:401] 127.19.183.212:59119/kms/v1/keys
Aug 09 12:19:39 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277978, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.183.212@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
Aug 09 12:19:39 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277978, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.19.183.212@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
Aug 09 12:19:41 dist-test-slave-39l8 krb5kdc[5090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:19:41.990293 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:44741
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:44741
--ranger_config_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.183.212:59119/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:19:42.335637 5756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:19:42.335949 5756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:19:42.336102 5756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:19:42.343133 5756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:19:42.343233 5756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:19:42.343308 5756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:19:42.343398 5756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:19:42.343480 5756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:19:42.354960 5756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/wal
--ranger_kms_url=127.19.183.212:59119/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:44741
--ranger_config_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:44741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:19:42.356290 5756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:19:42.357623 5756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:19:42.370551 5761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:19:42.370482 5762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:19:42.371411 5765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:19:42.372087 5756 server_base.cc:1034] running on GCE node
I20260809 12:19:42.372872 5756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:19:42.374303 5756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:19:42.375519 5756 hybrid_clock.cc:648] HybridClock initialized: now 1786277982375475 us; error 43 us; skew 500 ppm
I20260809 12:19:42.375867 5756 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:19:42 dist-test-slave-39l8 krb5kdc[5090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277982, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:19:42.382869 5756 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:19:42.384548 5756 webserver.cc:460] Webserver started at http://127.19.183.254:45041/ using document root <none> and password file <none>
I20260809 12:19:42.385406 5756 fs_manager.cc:359] Metadata directory not provided
I20260809 12:19:42.385550 5756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:19:42.385882 5756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:19:42 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277982, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
I20260809 12:19:42.440076 5756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "fd9e554e724e4b5a9d881b1659bb5def"
format_stamp: "Formatted at 2026-08-09 12:19:42 on dist-test-slave-39l8"
server_key: "fd81c0d9e95e1b92aae5d10a24d3913d"
server_key_iv: "a47891ced552ac5226f662a314d2647e"
server_key_version: "kuduclusterkey@0"
I20260809 12:19:42.440902 5756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "fd9e554e724e4b5a9d881b1659bb5def"
format_stamp: "Formatted at 2026-08-09 12:19:42 on dist-test-slave-39l8"
server_key: "fd81c0d9e95e1b92aae5d10a24d3913d"
server_key_iv: "a47891ced552ac5226f662a314d2647e"
server_key_version: "kuduclusterkey@0"
I20260809 12:19:42.446347 5756 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:19:42.466409 5773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:19:42.468055 5756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:19:42.468247 5756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "fd9e554e724e4b5a9d881b1659bb5def"
format_stamp: "Formatted at 2026-08-09 12:19:42 on dist-test-slave-39l8"
server_key: "fd81c0d9e95e1b92aae5d10a24d3913d"
server_key_iv: "a47891ced552ac5226f662a314d2647e"
server_key_version: "kuduclusterkey@0"
I20260809 12:19:42.468508 5756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:19:42.499871 5756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:19:42.504276 5756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:19:42.504632 5756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:19:42.524487 5756 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:44741
I20260809 12:19:42.524521 5847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:44741 every 8 connection(s)
I20260809 12:19:42.525870 5756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:19:42.529717 5849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:19:42.531134 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 5756
I20260809 12:19:42.531368 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:19:42.532090 20191 external_mini_cluster.cc:1428] Setting key d7abeaf3c37431b880cffb200ef9bb17
I20260809 12:19:42.541893 5849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def: Bootstrap starting.
I20260809 12:19:42.546027 5849 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def: Neither blocks nor log segments found. Creating new log.
Aug 09 12:19:42 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:19:42.547274 5849 log.cc:824] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def: Log is configured to *not* fsync() on all Append() calls
I20260809 12:19:42.551047 5849 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def: No bootstrap required, opened a new log
I20260809 12:19:42.556492 5854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:42.534068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51194 (local address 127.19.183.254:44741)
0809 12:19:42.534965 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:42.534978 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:42.535032 (+ 54us) server_negotiation.cc:407] Connection header received
0809 12:19:42.536711 (+ 1679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:42.536735 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:42.537112 (+ 377us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:42.537412 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:42.539184 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:42.540491 (+ 1307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:42.542406 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:42.543133 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:42.543272 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:42.548039 (+ 4767us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:42.548050 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:42.548066 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:42.548108 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:42.551876 (+ 3768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:42.552570 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:42.552578 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:42.552587 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:42.552805 (+ 218us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:42.553188 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:42.553193 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:42.553198 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:42.553919 (+ 721us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:42.554274 (+ 355us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:42.554626 (+ 352us) server_negotiation.cc:299] Negotiation successful
0809 12:19:42.555085 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":697,"thread_start_us":427,"threads_started":1}
I20260809 12:19:42.561069 5849 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9e554e724e4b5a9d881b1659bb5def" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44741 } }
I20260809 12:19:42.561406 5849 raft_consensus.cc:374] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:19:42.561481 5849 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd9e554e724e4b5a9d881b1659bb5def, State: Initialized, Role: FOLLOWER
I20260809 12:19:42.562021 5849 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9e554e724e4b5a9d881b1659bb5def" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44741 } }
I20260809 12:19:42.562235 5849 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:19:42.562332 5849 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:19:42.562458 5849 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:19:42.565045 5849 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9e554e724e4b5a9d881b1659bb5def" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44741 } }
I20260809 12:19:42.565584 5849 leader_election.cc:302] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd9e554e724e4b5a9d881b1659bb5def; no voters:
I20260809 12:19:42.566241 5849 leader_election.cc:288] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:19:42.566382 5857 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:19:42.567102 5857 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [term 1 LEADER]: Becoming Leader. State: Replica: fd9e554e724e4b5a9d881b1659bb5def, State: Running, Role: LEADER
I20260809 12:19:42.567693 5857 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9e554e724e4b5a9d881b1659bb5def" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44741 } }
I20260809 12:19:42.568428 5849 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:19:42.572108 5859 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fd9e554e724e4b5a9d881b1659bb5def" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9e554e724e4b5a9d881b1659bb5def" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44741 } } }
I20260809 12:19:42.572563 5859 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [sys.catalog]: This master's current role is: LEADER
I20260809 12:19:42.573199 5861 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd9e554e724e4b5a9d881b1659bb5def. Latest consensus state: current_term: 1 leader_uuid: "fd9e554e724e4b5a9d881b1659bb5def" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9e554e724e4b5a9d881b1659bb5def" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44741 } } }
I20260809 12:19:42.573553 5861 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [sys.catalog]: This master's current role is: LEADER
I20260809 12:19:42.575598 5876 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:19:42.579986 5876 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:19:42.580770 5849 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 12:19:42.601215 5876 catalog_manager.cc:1347] Generated new cluster ID: d4fc564beb674340ae74f5e8e9c14740
I20260809 12:19:42.601370 5876 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:19:42.619656 5876 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:19:42.622304 5876 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:19:42.637465 5876 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def: Generated new TSK 0
I20260809 12:19:42.638634 5876 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
I20260809 12:19:43.436169 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.183.212:59119/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:44741
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:19:43 dist-test-slave-39l8 krb5kdc[5090](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786277983, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 12:19:43.891773 5919 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:19:43.892215 5919 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:19:43.892422 5919 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:19:43.903635 5919 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:19:43.903841 5919 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:19:43.904045 5919 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:19:43.919744 5919 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.19.183.212:59119/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:44741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:19:43.921634 5919 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:19:43.923354 5919 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:19:43.939450 5936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:19:43.945457 5937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:19:43.945485 5939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:19:43.946748 5919 server_base.cc:1034] running on GCE node
I20260809 12:19:43.947417 5919 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:19:43.948382 5919 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:19:43.949625 5919 hybrid_clock.cc:648] HybridClock initialized: now 1786277983949564 us; error 54 us; skew 500 ppm
I20260809 12:19:43.949951 5919 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:19:43 dist-test-slave-39l8 krb5kdc[5090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277983, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:19:43.953378 5919 init.cc:377] Logged in from keytab as kudu/127.19.183.193@KRBTEST.COM (short username kudu)
I20260809 12:19:43.955158 5919 webserver.cc:460] Webserver started at http://127.19.183.193:41401/ using document root <none> and password file <none>
I20260809 12:19:43.956085 5919 fs_manager.cc:359] Metadata directory not provided
I20260809 12:19:43.956228 5919 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:19:43.956604 5919 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:19:43 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277983, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
I20260809 12:19:43.987483 5919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "bd56c3f50103472eb84aad2600b2351b"
format_stamp: "Formatted at 2026-08-09 12:19:43 on dist-test-slave-39l8"
server_key: "be7e752c1c668e5fd2a04485dddbea5e"
server_key_iv: "a8baa01d6ce83b099dce2139d740e24a"
server_key_version: "kuduclusterkey@0"
I20260809 12:19:43.988646 5919 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "bd56c3f50103472eb84aad2600b2351b"
format_stamp: "Formatted at 2026-08-09 12:19:43 on dist-test-slave-39l8"
server_key: "be7e752c1c668e5fd2a04485dddbea5e"
server_key_iv: "a8baa01d6ce83b099dce2139d740e24a"
server_key_version: "kuduclusterkey@0"
I20260809 12:19:43.996634 5919 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260809 12:19:44.037333 5949 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:19:44.039355 5919 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.006s sys 0.004s
I20260809 12:19:44.039640 5919 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "bd56c3f50103472eb84aad2600b2351b"
format_stamp: "Formatted at 2026-08-09 12:19:43 on dist-test-slave-39l8"
server_key: "be7e752c1c668e5fd2a04485dddbea5e"
server_key_iv: "a8baa01d6ce83b099dce2139d740e24a"
server_key_version: "kuduclusterkey@0"
I20260809 12:19:44.040001 5919 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:19:44.106189 5919 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:19:44.111698 5919 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:19:44.112179 5919 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:19:44.119632 5919 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:19:44.127887 5919 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:19:44.128064 5919 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260809 12:19:44.128296 5919 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:19:44.128448 5919 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:19:44 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277983, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
I20260809 12:19:44.474239 6107 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:43825 every 8 connection(s)
I20260809 12:19:44.474618 5919 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:43825
I20260809 12:19:44.476506 5919 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:19:44.484793 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 5919
I20260809 12:19:44.484999 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:19:44.485960 20191 external_mini_cluster.cc:1428] Setting key 94545f06364ca475f88a6eaff7f1c074
Aug 09 12:19:44 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277983, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:19:44.522694 6112 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:44.485999 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:44741 (local address 127.19.183.193:36249)
0809 12:19:44.496470 (+ 10471us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:44.496524 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 12:19:44.500485 (+ 3961us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:19:44.502869 (+ 2384us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:19:44.502881 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:19:44.503561 (+ 680us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:19:44.504361 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:44.504387 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:44.506001 (+ 1614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:44.506007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:44.508048 (+ 2041us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:44.508064 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:44.509327 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:44.509335 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:44.509619 (+ 284us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:44.510504 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:19:44.510535 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:19:44.513510 (+ 2975us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:19:44.517674 (+ 4164us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:44.517683 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:44.517700 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:44.518310 (+ 610us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:44.518837 (+ 527us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:44.518842 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:44.518845 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:44.519119 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:44.519923 (+ 804us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:19:44.519931 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:19:44.520367 (+ 436us) client_negotiation.cc:769] Sending connection context
0809 12:19:44.520728 (+ 361us) client_negotiation.cc:240] Negotiation successful
0809 12:19:44.521266 (+ 538us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10119,"thread_start_us":12575,"threads_started":1}
I20260809 12:19:44.523936 6111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:44.485342 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:36249 (local address 127.19.183.254:44741)
0809 12:19:44.501543 (+ 16201us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:44.501552 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:44.501575 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:19:44.501640 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:44.501645 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:44.501722 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:44.501854 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:44.504682 (+ 2828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:44.505779 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:44.508235 (+ 2456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:44.508868 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:44.508999 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:44.513775 (+ 4776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:44.513782 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:44.513788 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:44.513827 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:44.517473 (+ 3646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:44.518467 (+ 994us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:44.518473 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:44.518477 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:44.518652 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:44.519275 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:44.519280 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:44.519284 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:44.519675 (+ 391us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:44.519841 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:44.523307 (+ 3466us) server_negotiation.cc:299] Negotiation successful
0809 12:19:44.523540 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":16066,"thread_start_us":15752,"threads_started":1}
I20260809 12:19:44.526266 6109 heartbeater.cc:347] Connected to a master server at 127.19.183.254:44741
I20260809 12:19:44.526597 6109 heartbeater.cc:464] Registering TS with master...
I20260809 12:19:44.527508 6109 heartbeater.cc:511] Master 127.19.183.254:44741 requested a full tablet report, sending...
I20260809 12:19:44.530145 5794 ts_manager.cc:194] Registered new tserver with Master: bd56c3f50103472eb84aad2600b2351b (127.19.183.193:43825)
I20260809 12:19:44.532390 5794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:36249
WARNING: no policy specified for kudu/127.19.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
I20260809 12:19:44.604151 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.183.212:59119/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:44741
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:19:44 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277983, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
W20260809 12:19:45.091167 6116 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:19:45.091430 6116 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:19:45.091596 6116 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:19:45.098922 6116 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:19:45.099045 6116 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:19:45.099220 6116 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:19:45.108816 6116 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.19.183.212:59119/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:44741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:19:45.110136 6116 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:19:45.111487 6116 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:19:45.126149 6135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:19:45.126163 6134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:19:45.126875 6137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:19:45.132181 6116 server_base.cc:1034] running on GCE node
I20260809 12:19:45.132788 6116 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:19:45.133992 6116 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:19:45.135215 6116 hybrid_clock.cc:648] HybridClock initialized: now 1786277985135143 us; error 51 us; skew 500 ppm
I20260809 12:19:45.135530 6116 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:19:45 dist-test-slave-39l8 krb5kdc[5090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277985, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:19:45.138669 6116 init.cc:377] Logged in from keytab as kudu/127.19.183.194@KRBTEST.COM (short username kudu)
I20260809 12:19:45.140321 6116 webserver.cc:460] Webserver started at http://127.19.183.194:40999/ using document root <none> and password file <none>
I20260809 12:19:45.141014 6116 fs_manager.cc:359] Metadata directory not provided
I20260809 12:19:45.141150 6116 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:19:45.141465 6116 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:19:45 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277985, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
I20260809 12:19:45.159803 6116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "a025ca2df11a430490a2fc19937ef150"
format_stamp: "Formatted at 2026-08-09 12:19:45 on dist-test-slave-39l8"
server_key: "2d83a1aae8f40bbff9f149ec676ec624"
server_key_iv: "40a337381505308c56049eea2a221733"
server_key_version: "kuduclusterkey@0"
I20260809 12:19:45.160940 6116 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "a025ca2df11a430490a2fc19937ef150"
format_stamp: "Formatted at 2026-08-09 12:19:45 on dist-test-slave-39l8"
server_key: "2d83a1aae8f40bbff9f149ec676ec624"
server_key_iv: "40a337381505308c56049eea2a221733"
server_key_version: "kuduclusterkey@0"
I20260809 12:19:45.169266 6116 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.006s sys 0.000s
I20260809 12:19:45.188349 6147 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:19:45.189903 6116 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260809 12:19:45.190083 6116 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "a025ca2df11a430490a2fc19937ef150"
format_stamp: "Formatted at 2026-08-09 12:19:45 on dist-test-slave-39l8"
server_key: "2d83a1aae8f40bbff9f149ec676ec624"
server_key_iv: "40a337381505308c56049eea2a221733"
server_key_version: "kuduclusterkey@0"
I20260809 12:19:45.190328 6116 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:19:45.212728 6116 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:19:45.216993 6116 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:19:45.217305 6116 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:19:45.218365 6116 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:19:45.219949 6116 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:19:45.220054 6116 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:19:45.220201 6116 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:19:45.220261 6116 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:19:45.264540 6116 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:35717
I20260809 12:19:45.264560 6316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:35717 every 8 connection(s)
I20260809 12:19:45.265969 6116 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:19:45.266480 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 6116
I20260809 12:19:45.266819 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:19:45.267660 20191 external_mini_cluster.cc:1428] Setting key 07a98b80c2de2195d3db63c64d44ec0e
Aug 09 12:19:45 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277985, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:19:45.291976 6321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:45.270633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:50041 (local address 127.19.183.254:44741)
0809 12:19:45.275138 (+ 4505us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:45.275147 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:45.275167 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:19:45.275229 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:45.275237 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:45.275317 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:45.275430 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:45.277348 (+ 1918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:45.278532 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:45.280653 (+ 2121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:45.281329 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:45.281470 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:45.285485 (+ 4015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:45.285489 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:45.285492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:45.285522 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:45.288460 (+ 2938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:45.289406 (+ 946us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:45.289412 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:45.289415 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:45.289560 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:45.290055 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:45.290059 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:45.290061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:45.290364 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:45.290489 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:45.291385 (+ 896us) server_negotiation.cc:299] Negotiation successful
0809 12:19:45.291607 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4341,"thread_start_us":4220,"threads_started":1}
I20260809 12:19:45.293563 6320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:45.269747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:44741 (local address 127.19.183.194:50041)
0809 12:19:45.270634 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:45.270698 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 12:19:45.273982 (+ 3284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:19:45.275704 (+ 1722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:19:45.275727 (+ 23us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:19:45.276329 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:19:45.277144 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:45.277170 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:45.278745 (+ 1575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:45.278750 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:45.280489 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:45.280512 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:45.281568 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:45.281574 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:45.281840 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:45.282571 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:19:45.282620 (+ 49us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:19:45.285274 (+ 2654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:19:45.288613 (+ 3339us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:45.288632 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:45.288648 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:45.289267 (+ 619us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:45.289678 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:45.289684 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:45.289690 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:45.289941 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:45.290501 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:19:45.290511 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:19:45.290960 (+ 449us) client_negotiation.cc:769] Sending connection context
0809 12:19:45.291261 (+ 301us) client_negotiation.cc:240] Negotiation successful
0809 12:19:45.291778 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":367,"threads_started":1}
I20260809 12:19:45.295163 6317 heartbeater.cc:347] Connected to a master server at 127.19.183.254:44741
I20260809 12:19:45.295481 6317 heartbeater.cc:464] Registering TS with master...
I20260809 12:19:45.296301 6317 heartbeater.cc:511] Master 127.19.183.254:44741 requested a full tablet report, sending...
I20260809 12:19:45.298022 5794 ts_manager.cc:194] Registered new tserver with Master: a025ca2df11a430490a2fc19937ef150 (127.19.183.194:35717)
I20260809 12:19:45.299320 5794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:50041
WARNING: no policy specified for kudu/127.19.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
I20260809 12:19:45.335984 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.183.212:59119/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:44741
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:19:45.536391 6109 heartbeater.cc:503] Master 127.19.183.254:44741 was elected leader, sending a full tablet report...
W20260809 12:19:45.674188 6325 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:19:45.674458 6325 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:19:45.674625 6325 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:19:45.681427 6325 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:19:45.681542 6325 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:19:45.681706 6325 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:19:45.691108 6325 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.19.183.212:59119/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:44741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:19:45.692354 6325 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:19:45.693626 6325 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:19:45.706385 6334 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:19:45.706485 6335 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:19:45.706733 6338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:19:45.707497 6325 server_base.cc:1034] running on GCE node
I20260809 12:19:45.708021 6325 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:19:45.708920 6325 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:19:45.710143 6325 hybrid_clock.cc:648] HybridClock initialized: now 1786277985710098 us; error 39 us; skew 500 ppm
I20260809 12:19:45.710433 6325 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:19:45 dist-test-slave-39l8 krb5kdc[5090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277985, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:19:45.713611 6325 init.cc:377] Logged in from keytab as kudu/127.19.183.195@KRBTEST.COM (short username kudu)
I20260809 12:19:45.715139 6325 webserver.cc:460] Webserver started at http://127.19.183.195:35971/ using document root <none> and password file <none>
I20260809 12:19:45.715793 6325 fs_manager.cc:359] Metadata directory not provided
I20260809 12:19:45.715931 6325 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:19:45.716243 6325 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 12:19:45 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277985, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
I20260809 12:19:45.730844 6325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "02d460f29a094bcebb52310a6a8616c2"
format_stamp: "Formatted at 2026-08-09 12:19:45 on dist-test-slave-39l8"
server_key: "a2e76dc02a98bc3c3c90f82b9b36870b"
server_key_iv: "8770421f7dcef25bbe15bdf20d206e61"
server_key_version: "kuduclusterkey@0"
I20260809 12:19:45.731817 6325 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "02d460f29a094bcebb52310a6a8616c2"
format_stamp: "Formatted at 2026-08-09 12:19:45 on dist-test-slave-39l8"
server_key: "a2e76dc02a98bc3c3c90f82b9b36870b"
server_key_iv: "8770421f7dcef25bbe15bdf20d206e61"
server_key_version: "kuduclusterkey@0"
I20260809 12:19:45.737459 6325 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:19:45.753474 6348 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:19:45.755175 6325 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:19:45.755357 6325 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "02d460f29a094bcebb52310a6a8616c2"
format_stamp: "Formatted at 2026-08-09 12:19:45 on dist-test-slave-39l8"
server_key: "a2e76dc02a98bc3c3c90f82b9b36870b"
server_key_iv: "8770421f7dcef25bbe15bdf20d206e61"
server_key_version: "kuduclusterkey@0"
I20260809 12:19:45.755641 6325 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:19:45.781689 6325 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:19:45.785354 6325 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:19:45.785611 6325 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:19:45.786451 6325 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:19:45.787782 6325 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:19:45.787855 6325 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:19:45.787937 6325 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:19:45.787981 6325 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:19:45.820065 6325 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:41617
I20260809 12:19:45.820101 6521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:41617 every 8 connection(s)
I20260809 12:19:45.821331 6325 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:19:45.822065 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 6325
I20260809 12:19:45.822261 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:19:45.822819 20191 external_mini_cluster.cc:1428] Setting key 88cd47ea00b2961616bad201b11cad21
Aug 09 12:19:45 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277985, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:19:45.842933 6528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:45.825988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:46907 (local address 127.19.183.254:44741)
0809 12:19:45.826492 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:45.826499 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:45.827050 (+ 551us) server_negotiation.cc:407] Connection header received
0809 12:19:45.828757 (+ 1707us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:45.828762 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:45.828854 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:45.828961 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:45.830418 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:45.831167 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:45.832769 (+ 1602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:45.833215 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:45.833343 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:45.836565 (+ 3222us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:45.836569 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:45.836572 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:45.836621 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:45.839948 (+ 3327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:45.840658 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:45.840664 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:45.840668 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:45.840797 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:45.841260 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:45.841265 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:45.841269 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:45.841548 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:45.841710 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:45.842328 (+ 618us) server_negotiation.cc:299] Negotiation successful
0809 12:19:45.842536 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":289,"threads_started":1}
I20260809 12:19:45.843865 6527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:45.825429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:44741 (local address 127.19.183.195:46907)
0809 12:19:45.826875 (+ 1446us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:45.826917 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 12:19:45.828512 (+ 1595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:19:45.829188 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:19:45.829196 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:19:45.829660 (+ 464us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:19:45.830254 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:45.830271 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:45.831332 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:45.831336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:45.832619 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:45.832640 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:45.833442 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:45.833450 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:45.833690 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:45.834315 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:19:45.834337 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:19:45.836343 (+ 2006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:19:45.840095 (+ 3752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:45.840103 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:45.840119 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:45.840519 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:45.840913 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:45.840917 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:45.840921 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:45.841149 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:45.841686 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:19:45.841693 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:19:45.842011 (+ 318us) client_negotiation.cc:769] Sending connection context
0809 12:19:45.842255 (+ 244us) client_negotiation.cc:240] Negotiation successful
0809 12:19:45.842795 (+ 540us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1135,"thread_start_us":918,"threads_started":1}
I20260809 12:19:45.845294 6522 heartbeater.cc:347] Connected to a master server at 127.19.183.254:44741
I20260809 12:19:45.845592 6522 heartbeater.cc:464] Registering TS with master...
I20260809 12:19:45.846282 6522 heartbeater.cc:511] Master 127.19.183.254:44741 requested a full tablet report, sending...
I20260809 12:19:45.847783 5794 ts_manager.cc:194] Registered new tserver with Master: 02d460f29a094bcebb52310a6a8616c2 (127.19.183.195:41617)
I20260809 12:19:45.848572 5794 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:46907
I20260809 12:19:45.851580 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:19:45.853279 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 5919
Aug 09 12:19:45 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.193@KRBTEST.COM
I20260809 12:19:45.869051 6531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:45.854248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54582 (local address 127.19.183.193:43825)
0809 12:19:45.854921 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:45.854928 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:45.854950 (+ 22us) server_negotiation.cc:407] Connection header received
0809 12:19:45.855011 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:45.855017 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:45.855178 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:45.855294 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:45.856629 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:45.857727 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:45.859139 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:45.859759 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:45.859868 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:45.864022 (+ 4154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:45.864034 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:45.864047 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:45.864080 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:45.866558 (+ 2478us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:45.867230 (+ 672us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:45.867237 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:45.867239 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:45.867378 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:45.867682 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:45.867686 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:45.867687 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:45.867988 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:45.868218 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:45.868498 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 12:19:45.868693 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":537,"thread_start_us":343,"threads_started":1}
I20260809 12:19:45.870296 6101 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:19:45.950201 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 5919
I20260809 12:19:45.976296 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 6116
Aug 09 12:19:45 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.194@KRBTEST.COM
I20260809 12:19:45.991562 6533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:45.977212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45892 (local address 127.19.183.194:35717)
0809 12:19:45.977770 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:45.977778 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:45.977809 (+ 31us) server_negotiation.cc:407] Connection header received
0809 12:19:45.977872 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:45.977878 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:45.978094 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:45.978225 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:45.979613 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:45.980591 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:45.981951 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:45.982462 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:45.982554 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:45.986784 (+ 4230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:45.986798 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:45.986817 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:45.986848 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:45.989359 (+ 2511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:45.989929 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:45.989935 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:45.989937 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:45.990053 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:45.990340 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:45.990342 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:45.990344 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:45.990632 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:45.990900 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:45.991096 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 12:19:45.991285 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":262,"threads_started":1}
I20260809 12:19:45.992867 6309 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:19:46.079248 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 6116
I20260809 12:19:46.106828 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 6325
Aug 09 12:19:46 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277981, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.195@KRBTEST.COM
I20260809 12:19:46.122216 6537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:46.107807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59408 (local address 127.19.183.195:41617)
0809 12:19:46.108423 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:46.108430 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:46.108459 (+ 29us) server_negotiation.cc:407] Connection header received
0809 12:19:46.108529 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:46.108535 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:46.108741 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:46.108870 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:46.110152 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:46.111193 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:46.112409 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:46.112932 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:46.113027 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:46.117331 (+ 4304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:46.117340 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:46.117353 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:46.117385 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:46.119897 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:46.120450 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:46.120456 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:46.120458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:46.120580 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:46.120869 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:46.120872 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:46.120874 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:46.121213 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:46.121435 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:46.121644 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 12:19:46.121827 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":456,"thread_start_us":289,"threads_started":1}
I20260809 12:19:46.123543 6513 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:19:46.204346 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 6325
I20260809 12:19:46.231710 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 5756
I20260809 12:19:46.233047 5840 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:19:46.336441 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 5756
I20260809 12:19:46.368085 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:44741
--webserver_interface=127.19.183.254
--webserver_port=45041
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:44741
--ranger_config_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.183.212:59119/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.183.212:59119/kms with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:19:46.693028 6540 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:19:46.693410 6540 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:19:46.693571 6540 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:19:46.700332 6540 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:19:46.700466 6540 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:19:46.700582 6540 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:19:46.700686 6540 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:19:46.700755 6540 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:19:46.710462 6540 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.183.212:59119/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:44741
--ranger_config_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:44741
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=45041
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:19:46.711689 6540 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:19:46.712997 6540 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:19:46.724268 6547 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:19:46.724334 6548 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:19:46.724310 6550 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:19:46.725580 6540 server_base.cc:1034] running on GCE node
I20260809 12:19:46.726526 6540 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:19:46.727931 6540 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:19:46.729159 6540 hybrid_clock.cc:648] HybridClock initialized: now 1786277986729121 us; error 44 us; skew 500 ppm
I20260809 12:19:46.729502 6540 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:19:46 dist-test-slave-39l8 krb5kdc[5090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277986, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:19:46.732575 6540 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:19:46.734050 6540 webserver.cc:460] Webserver started at http://127.19.183.254:45041/ using document root <none> and password file <none>
I20260809 12:19:46.734673 6540 fs_manager.cc:359] Metadata directory not provided
I20260809 12:19:46.734881 6540 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 12:19:46 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277986, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
I20260809 12:19:46.785373 6540 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 12:19:46.788673 6561 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:19:46.790143 6540 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:19:46.790328 6540 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "fd9e554e724e4b5a9d881b1659bb5def"
format_stamp: "Formatted at 2026-08-09 12:19:42 on dist-test-slave-39l8"
server_key: "fd81c0d9e95e1b92aae5d10a24d3913d"
server_key_iv: "a47891ced552ac5226f662a314d2647e"
server_key_version: "kuduclusterkey@0"
I20260809 12:19:46.791376 6540 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:19:46.805987 6540 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:19:46.809487 6540 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:19:46.809765 6540 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:19:46.828015 6540 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:44741
I20260809 12:19:46.828048 6624 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:44741 every 8 connection(s)
I20260809 12:19:46.829455 6540 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:19:46.835676 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 6540
I20260809 12:19:46.836532 6628 sys_catalog.cc:263] Verifying existing consensus state
I20260809 12:19:46.836747 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.183.212:59119/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:43825
--local_ip_for_outbound_sockets=127.19.183.193
--tserver_master_addrs=127.19.183.254:44741
--webserver_port=41401
--webserver_interface=127.19.183.193
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.183.212:59119/kms with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:19:46.841571 6628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def: Bootstrap starting.
I20260809 12:19:46.862509 6628 log.cc:824] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def: Log is configured to *not* fsync() on all Append() calls
I20260809 12:19:46.870865 6628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:19:46.871341 6628 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def: Bootstrap complete.
I20260809 12:19:46.880234 6628 raft_consensus.cc:348] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9e554e724e4b5a9d881b1659bb5def" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44741 } }
I20260809 12:19:46.881023 6628 raft_consensus.cc:729] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd9e554e724e4b5a9d881b1659bb5def, State: Initialized, Role: FOLLOWER
I20260809 12:19:46.881572 6628 consensus_queue.cc:260] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9e554e724e4b5a9d881b1659bb5def" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44741 } }
I20260809 12:19:46.881801 6628 raft_consensus.cc:388] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:19:46.881925 6628 raft_consensus.cc:482] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:19:46.882086 6628 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [term 1 FOLLOWER]: Advancing to term 2
I20260809 12:19:46.885686 6628 raft_consensus.cc:504] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9e554e724e4b5a9d881b1659bb5def" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44741 } }
I20260809 12:19:46.886163 6628 leader_election.cc:302] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fd9e554e724e4b5a9d881b1659bb5def; no voters:
I20260809 12:19:46.886854 6628 leader_election.cc:288] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 12:19:46.886989 6632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [term 2 FOLLOWER]: Leader election won for term 2
I20260809 12:19:46.888266 6632 raft_consensus.cc:686] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [term 2 LEADER]: Becoming Leader. State: Replica: fd9e554e724e4b5a9d881b1659bb5def, State: Running, Role: LEADER
I20260809 12:19:46.888875 6632 consensus_queue.cc:237] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9e554e724e4b5a9d881b1659bb5def" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44741 } }
I20260809 12:19:46.889081 6628 sys_catalog.cc:564] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:19:46.891789 6636 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "fd9e554e724e4b5a9d881b1659bb5def" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9e554e724e4b5a9d881b1659bb5def" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44741 } } }
I20260809 12:19:46.891822 6637 sys_catalog.cc:455] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [sys.catalog]: SysCatalogTable state changed. Reason: New leader fd9e554e724e4b5a9d881b1659bb5def. Latest consensus state: current_term: 2 leader_uuid: "fd9e554e724e4b5a9d881b1659bb5def" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fd9e554e724e4b5a9d881b1659bb5def" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44741 } } }
I20260809 12:19:46.892352 6636 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [sys.catalog]: This master's current role is: LEADER
I20260809 12:19:46.892345 6637 sys_catalog.cc:458] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def [sys.catalog]: This master's current role is: LEADER
I20260809 12:19:46.897091 6643 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:19:46.903883 6643 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:19:46.905665 6643 catalog_manager.cc:1259] Loaded cluster ID: d4fc564beb674340ae74f5e8e9c14740
I20260809 12:19:46.905844 6643 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:19:46.910722 6643 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:19:46.912505 6643 catalog_manager.cc:5879] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def: Loaded TSK: 0
I20260809 12:19:46.914397 6643 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:19:46.918735 6628 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
W20260809 12:19:47.291636 6630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:19:47.292084 6630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:19:47.292302 6630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:19:47.304157 6630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:19:47.304359 6630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:19:47.304572 6630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:19:47.320565 6630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.183.212:59119/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:43825
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=41401
--tserver_master_addrs=127.19.183.254:44741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:19:47.322504 6630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:19:47.324397 6630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:19:47.363049 6695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:19:47.371124 6699 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:19:47.371636 6630 server_base.cc:1034] running on GCE node
W20260809 12:19:47.373901 6694 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:19:47.374933 6630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:19:47.381992 6630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:19:47.383307 6630 hybrid_clock.cc:648] HybridClock initialized: now 1786277987383232 us; error 65 us; skew 500 ppm
I20260809 12:19:47.383802 6630 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:19:47 dist-test-slave-39l8 krb5kdc[5090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277987, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:19:47.392982 6630 init.cc:377] Logged in from keytab as kudu/127.19.183.193@KRBTEST.COM (short username kudu)
I20260809 12:19:47.412956 6630 webserver.cc:460] Webserver started at http://127.19.183.193:41401/ using document root <none> and password file <none>
I20260809 12:19:47.414108 6630 fs_manager.cc:359] Metadata directory not provided
I20260809 12:19:47.414353 6630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 12:19:47 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277987, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
I20260809 12:19:47.448985 6630 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.004s sys 0.000s
I20260809 12:19:47.458880 6712 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:19:47.460795 6630 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.003s sys 0.005s
I20260809 12:19:47.461043 6630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "bd56c3f50103472eb84aad2600b2351b"
format_stamp: "Formatted at 2026-08-09 12:19:43 on dist-test-slave-39l8"
server_key: "be7e752c1c668e5fd2a04485dddbea5e"
server_key_iv: "a8baa01d6ce83b099dce2139d740e24a"
server_key_version: "kuduclusterkey@0"
I20260809 12:19:47.462531 6630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:19:47.498931 6630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:19:47.503954 6630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:19:47.504271 6630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:19:47.505393 6630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:19:47.506950 6630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:19:47.507076 6630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:19:47.507201 6630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:19:47.507292 6630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260809 12:19:47.579442 6630 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:43825
I20260809 12:19:47.579550 6872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:43825 every 8 connection(s)
I20260809 12:19:47.581013 6630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:19:47.587975 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 6630
I20260809 12:19:47.588791 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.183.212:59119/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:35717
--local_ip_for_outbound_sockets=127.19.183.194
--tserver_master_addrs=127.19.183.254:44741
--webserver_port=40999
--webserver_interface=127.19.183.194
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.183.212:59119/kms with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:19:47 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277987, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:19:47.639802 6875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:47.584969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:37711 (local address 127.19.183.254:44741)
0809 12:19:47.615678 (+ 30709us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:47.615704 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:47.615774 (+ 70us) server_negotiation.cc:407] Connection header received
0809 12:19:47.616318 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:47.616343 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:47.616968 (+ 625us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:47.617831 (+ 863us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:47.620085 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:47.622059 (+ 1974us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:47.624710 (+ 2651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:47.625710 (+ 1000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:47.625899 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:47.630510 (+ 4611us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:47.630520 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:47.630537 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:47.630581 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:47.634700 (+ 4119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:47.635655 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:47.635663 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:47.635671 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:47.635862 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:47.636421 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:47.636427 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:47.636430 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:47.637032 (+ 602us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:47.637240 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:47.638174 (+ 934us) server_negotiation.cc:299] Negotiation successful
0809 12:19:47.638570 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30346,"thread_start_us":32289,"threads_started":1}
I20260809 12:19:47.639809 6876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:47.585137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:44741 (local address 127.19.183.193:37711)
0809 12:19:47.587464 (+ 2327us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:47.587526 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 12:19:47.589506 (+ 1980us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:19:47.618105 (+ 28599us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:19:47.618120 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:19:47.618917 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:19:47.619822 (+ 905us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:47.619857 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:47.622282 (+ 2425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:47.622289 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:47.624482 (+ 2193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:47.624499 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:47.626302 (+ 1803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:47.626311 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:47.626614 (+ 303us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:47.627495 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:19:47.627538 (+ 43us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:19:47.630233 (+ 2695us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:19:47.634883 (+ 4650us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:47.634892 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:47.634906 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:47.635471 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:47.636002 (+ 531us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:47.636007 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:47.636011 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:47.636276 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:47.637196 (+ 920us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:19:47.637205 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:19:47.637601 (+ 396us) client_negotiation.cc:769] Sending connection context
0809 12:19:47.637911 (+ 310us) client_negotiation.cc:240] Negotiation successful
0809 12:19:47.638462 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2059,"thread_start_us":1835,"threads_started":1}
I20260809 12:19:47.644048 6873 heartbeater.cc:347] Connected to a master server at 127.19.183.254:44741
I20260809 12:19:47.644373 6873 heartbeater.cc:464] Registering TS with master...
I20260809 12:19:47.645259 6873 heartbeater.cc:511] Master 127.19.183.254:44741 requested a full tablet report, sending...
W20260809 12:19:47.648765 6873 heartbeater.cc:644] Failed to heartbeat to 127.19.183.254:44741 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 12:19:47.648996 6873 heartbeater.cc:464] Registering TS with master...
I20260809 12:19:47.649272 6873 heartbeater.cc:511] Master 127.19.183.254:44741 requested a full tablet report, sending...
I20260809 12:19:47.650627 6873 heartbeater.cc:464] Registering TS with master...
I20260809 12:19:47.650981 6873 heartbeater.cc:511] Master 127.19.183.254:44741 requested a full tablet report, sending...
W20260809 12:19:47.652139 6873 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 12:19:48.042088 6877 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:19:48.042582 6877 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:19:48.042820 6877 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:19:48.051838 6877 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:19:48.051983 6877 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:19:48.052206 6877 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:19:48.062016 6877 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.183.212:59119/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:35717
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=40999
--tserver_master_addrs=127.19.183.254:44741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:19:48.063540 6877 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:19:48.065001 6877 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:19:48.082873 6901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:19:48.084316 6902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:19:48.092204 6904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:19:48.093016 6877 server_base.cc:1034] running on GCE node
I20260809 12:19:48.093750 6877 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:19:48.094957 6877 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:19:48.096217 6877 hybrid_clock.cc:648] HybridClock initialized: now 1786277988096155 us; error 63 us; skew 500 ppm
I20260809 12:19:48.096613 6877 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:19:48 dist-test-slave-39l8 krb5kdc[5090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277988, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:19:48.100376 6877 init.cc:377] Logged in from keytab as kudu/127.19.183.194@KRBTEST.COM (short username kudu)
I20260809 12:19:48.104220 6877 webserver.cc:460] Webserver started at http://127.19.183.194:40999/ using document root <none> and password file <none>
I20260809 12:19:48.105250 6877 fs_manager.cc:359] Metadata directory not provided
I20260809 12:19:48.105417 6877 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 12:19:48 dist-test-slave-39l8 krb5kdc[5090](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786277988, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 12:19:48 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277988, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
I20260809 12:19:48.148777 6877 fs_manager.cc:709] Time spent opening directory manager: real 0.015s user 0.010s sys 0.003s
I20260809 12:19:48.152680 6913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:19:48.154439 6877 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260809 12:19:48.154614 6877 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "a025ca2df11a430490a2fc19937ef150"
format_stamp: "Formatted at 2026-08-09 12:19:45 on dist-test-slave-39l8"
server_key: "2d83a1aae8f40bbff9f149ec676ec624"
server_key_iv: "40a337381505308c56049eea2a221733"
server_key_version: "kuduclusterkey@0"
I20260809 12:19:48.156000 6877 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:19:48.184111 6877 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:19:48.189296 6877 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:19:48.189615 6877 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:19:48.190985 6877 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:19:48.192901 6877 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:19:48.193046 6877 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:19:48.193172 6877 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:19:48.193259 6877 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:19:48.243034 6877 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:35717
I20260809 12:19:48.243115 7082 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:35717 every 8 connection(s)
I20260809 12:19:48.245100 6877 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:19:48.252542 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 6877
I20260809 12:19:48.253587 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.19.183.212:59119/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:41617
--local_ip_for_outbound_sockets=127.19.183.195
--tserver_master_addrs=127.19.183.254:44741
--webserver_port=35971
--webserver_interface=127.19.183.195
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40781
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.19.183.212:59119/kms with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:19:48 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277988, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:19:48.290422 7088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:48.249774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:53379 (local address 127.19.183.254:44741)
0809 12:19:48.265354 (+ 15580us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:48.265361 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:48.265392 (+ 31us) server_negotiation.cc:407] Connection header received
0809 12:19:48.269959 (+ 4567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:48.269966 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:48.270057 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:48.270198 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:48.272466 (+ 2268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:48.273640 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:48.276139 (+ 2499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:48.276775 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:48.276887 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:48.281452 (+ 4565us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:48.281459 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:48.281464 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:48.281501 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:48.285043 (+ 3542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:48.285902 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:48.285907 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:48.285910 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:48.286048 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:48.286571 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:48.286576 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:48.286579 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:48.286962 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:48.287104 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:48.289594 (+ 2490us) server_negotiation.cc:299] Negotiation successful
0809 12:19:48.289831 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15420,"thread_start_us":15612,"threads_started":1}
I20260809 12:19:48.292801 7089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:48.250442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:44741 (local address 127.19.183.194:53379)
0809 12:19:48.254514 (+ 4072us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:48.254593 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0809 12:19:48.268807 (+ 14214us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:19:48.270728 (+ 1921us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:19:48.270742 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:19:48.271347 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:19:48.272225 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:48.272253 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:48.273873 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:48.273879 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:48.275959 (+ 2080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:48.275975 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:48.277239 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:48.277247 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:48.277552 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:48.278385 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:19:48.278417 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:19:48.281186 (+ 2769us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:19:48.285213 (+ 4027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:48.285221 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:48.285237 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:48.285763 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:48.286179 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:48.286184 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:48.286187 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:48.286436 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:48.288304 (+ 1868us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:19:48.288314 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:19:48.288751 (+ 437us) client_negotiation.cc:769] Sending connection context
0809 12:19:48.289102 (+ 351us) client_negotiation.cc:240] Negotiation successful
0809 12:19:48.291227 (+ 2125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3667,"thread_start_us":3415,"threads_started":1}
I20260809 12:19:48.294790 7084 heartbeater.cc:347] Connected to a master server at 127.19.183.254:44741
I20260809 12:19:48.295044 7084 heartbeater.cc:464] Registering TS with master...
I20260809 12:19:48.296022 7084 heartbeater.cc:511] Master 127.19.183.254:44741 requested a full tablet report, sending...
I20260809 12:19:48.298396 6582 ts_manager.cc:194] Registered new tserver with Master: a025ca2df11a430490a2fc19937ef150 (127.19.183.194:35717)
I20260809 12:19:48.300825 6582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:53379
Aug 09 12:19:48 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277988, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
I20260809 12:19:48.652297 6873 heartbeater.cc:464] Registering TS with master...
I20260809 12:19:48.652804 6873 heartbeater.cc:511] Master 127.19.183.254:44741 requested a full tablet report, sending...
I20260809 12:19:48.654193 6582 ts_manager.cc:194] Registered new tserver with Master: bd56c3f50103472eb84aad2600b2351b (127.19.183.193:43825)
I20260809 12:19:48.655525 6582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:37711
W20260809 12:19:48.696825 7090 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:19:48.697176 7090 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:19:48.697314 7090 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:19:48.707402 7090 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:19:48.707564 7090 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:19:48.707752 7090 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:19:48.722254 7090 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40781
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.19.183.212:59119/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:41617
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=35971
--tserver_master_addrs=127.19.183.254:44741
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:19:48.724211 7090 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:19:48.726145 7090 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:19:48.745373 7097 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:19:48.747109 7095 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:19:48.758273 7099 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:19:48.759295 7090 server_base.cc:1034] running on GCE node
I20260809 12:19:48.760185 7090 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:19:48.766989 7090 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:19:48.768236 7090 hybrid_clock.cc:648] HybridClock initialized: now 1786277988768166 us; error 58 us; skew 500 ppm
I20260809 12:19:48.768613 7090 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:19:48 dist-test-slave-39l8 krb5kdc[5090](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277988, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:19:48.772975 7090 init.cc:377] Logged in from keytab as kudu/127.19.183.195@KRBTEST.COM (short username kudu)
I20260809 12:19:48.776299 7090 webserver.cc:460] Webserver started at http://127.19.183.195:35971/ using document root <none> and password file <none>
I20260809 12:19:48.777279 7090 fs_manager.cc:359] Metadata directory not provided
I20260809 12:19:48.777446 7090 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 12:19:48 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277988, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
I20260809 12:19:48.814913 7090 fs_manager.cc:709] Time spent opening directory manager: real 0.012s user 0.009s sys 0.000s
I20260809 12:19:48.826045 7114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:19:48.828050 7090 fs_manager.cc:725] Time spent opening block manager: real 0.011s user 0.003s sys 0.007s
I20260809 12:19:48.828310 7090 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "02d460f29a094bcebb52310a6a8616c2"
format_stamp: "Formatted at 2026-08-09 12:19:45 on dist-test-slave-39l8"
server_key: "a2e76dc02a98bc3c3c90f82b9b36870b"
server_key_iv: "8770421f7dcef25bbe15bdf20d206e61"
server_key_version: "kuduclusterkey@0"
I20260809 12:19:48.829778 7090 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:19:48.865262 7090 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:19:48.870812 7090 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:19:48.871202 7090 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:19:48.872958 7090 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:19:48.875208 7090 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:19:48.875324 7090 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:19:48.875411 7090 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:19:48.875474 7090 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 12:19:48 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277988, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for HTTP/127.19.183.212@KRBTEST.COM
I20260809 12:19:49.100469 7090 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:41617
I20260809 12:19:49.100528 7277 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:41617 every 8 connection(s)
I20260809 12:19:49.102648 7090 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:19:49.109663 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 7090
Aug 09 12:19:49 dist-test-slave-39l8 krb5kdc[5090](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277988, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:19:49.193940 7283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.110889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51210 (local address 127.19.183.254:44741)
0809 12:19:49.123328 (+ 12439us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:49.123339 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:49.158172 (+ 34833us) server_negotiation.cc:407] Connection header received
0809 12:19:49.158235 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:49.158240 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:49.158322 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:49.158422 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:49.160971 (+ 2549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.162067 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.169636 (+ 7569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.170272 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.170415 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:49.179255 (+ 8840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:49.179263 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:49.179268 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:49.179320 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:49.183074 (+ 3754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:49.188765 (+ 5691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:49.188770 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:49.188773 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:49.188948 (+ 175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:49.190292 (+ 1344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:49.190297 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:49.190300 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:49.190753 (+ 453us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:49.191036 (+ 283us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:49.193259 (+ 2223us) server_negotiation.cc:299] Negotiation successful
0809 12:19:49.193534 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12245,"thread_start_us":12464,"threads_started":1}
I20260809 12:19:49.199291 7284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.127130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:52305 (local address 127.19.183.254:44741)
0809 12:19:49.156177 (+ 29047us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:49.156184 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:49.156210 (+ 26us) server_negotiation.cc:407] Connection header received
0809 12:19:49.156284 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:49.156290 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:49.156410 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:49.156609 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:49.166360 (+ 9751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.167726 (+ 1366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.171332 (+ 3606us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.171961 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.172117 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:49.183350 (+ 11233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:49.183369 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:49.183374 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:49.183424 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:49.186906 (+ 3482us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:49.189464 (+ 2558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:49.189470 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:49.189473 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:49.189660 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:49.191594 (+ 1934us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:49.191600 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:49.191603 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:49.191980 (+ 377us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:49.192141 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:49.198555 (+ 6414us) server_negotiation.cc:299] Negotiation successful
0809 12:19:49.198888 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28885,"thread_start_us":29603,"threads_started":1}
I20260809 12:19:49.201936 7285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.126557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:44741 (local address 127.19.183.195:52305)
0809 12:19:49.150574 (+ 24017us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:49.150653 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0809 12:19:49.153708 (+ 3055us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:19:49.164482 (+ 10774us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:19:49.164496 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:19:49.165217 (+ 721us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:19:49.166100 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:49.166131 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.167975 (+ 1844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.167981 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:49.171142 (+ 3161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:49.171171 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.172497 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.172505 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:49.172819 (+ 314us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:49.173674 (+ 855us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:19:49.173713 (+ 39us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:19:49.178958 (+ 5245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:19:49.187903 (+ 8945us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:49.187913 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:49.187932 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:49.188579 (+ 647us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:49.189833 (+ 1254us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:49.189838 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:49.189842 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:49.190127 (+ 285us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:49.192258 (+ 2131us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:19:49.192268 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:19:49.192789 (+ 521us) client_negotiation.cc:769] Sending connection context
0809 12:19:49.199845 (+ 7056us) client_negotiation.cc:240] Negotiation successful
0809 12:19:49.200480 (+ 635us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23601,"thread_start_us":26842,"threads_started":1}
I20260809 12:19:49.203853 7279 heartbeater.cc:347] Connected to a master server at 127.19.183.254:44741
I20260809 12:19:49.204185 7279 heartbeater.cc:464] Registering TS with master...
I20260809 12:19:49.205158 7279 heartbeater.cc:511] Master 127.19.183.254:44741 requested a full tablet report, sending...
I20260809 12:19:49.206936 6582 ts_manager.cc:194] Registered new tserver with Master: 02d460f29a094bcebb52310a6a8616c2 (127.19.183.195:41617)
I20260809 12:19:49.208161 6582 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:52305
I20260809 12:19:49.211530 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:19:49.238693 7284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.220697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51224 (local address 127.19.183.254:44741)
0809 12:19:49.220863 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:49.220869 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:49.221441 (+ 572us) server_negotiation.cc:407] Connection header received
0809 12:19:49.221618 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:49.221623 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:49.221705 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:49.221810 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:49.223868 (+ 2058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.224956 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.226992 (+ 2036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.227620 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.227741 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:49.231143 (+ 3402us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:49.231152 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:49.231160 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:49.231205 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:49.235339 (+ 4134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:49.236282 (+ 943us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:49.236287 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:49.236291 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:49.236440 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:49.236908 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:49.236914 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:49.236917 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:49.237283 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:49.237511 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:49.238092 (+ 581us) server_negotiation.cc:299] Negotiation successful
0809 12:19:49.238334 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 12:19:49.245779 6582 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:19:49.255762 6582 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:19:49.287494 7314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.274129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45908 (local address 127.19.183.194:35717)
0809 12:19:49.274777 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:49.274785 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:49.274815 (+ 30us) server_negotiation.cc:407] Connection header received
0809 12:19:49.275148 (+ 333us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:49.275156 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:49.275402 (+ 246us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:49.275621 (+ 219us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:19:49.277704 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.279393 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.283342 (+ 3949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.284825 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.284994 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:49.285163 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:49.286991 (+ 1828us) server_negotiation.cc:299] Negotiation successful
0809 12:19:49.287163 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":1178,"threads_started":1}
I20260809 12:19:49.288241 7313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.273497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:35717 (local address 127.0.0.1:45908)
0809 12:19:49.274580 (+ 1083us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:49.274630 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 12:19:49.274848 (+ 218us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:19:49.275670 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:19:49.275681 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:19:49.275725 (+ 44us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:19:49.277461 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:49.277480 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.279573 (+ 2093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.279578 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:49.283149 (+ 3571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:49.283164 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.286450 (+ 3286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.286457 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:49.286681 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:49.286744 (+ 63us) client_negotiation.cc:769] Sending connection context
0809 12:19:49.287696 (+ 952us) client_negotiation.cc:240] Negotiation successful
0809 12:19:49.287826 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":828,"mutex_wait_us":9,"thread_start_us":856,"threads_started":1}
I20260809 12:19:49.288564 7315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.275192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:41617 (local address 127.0.0.1:59424)
0809 12:19:49.276331 (+ 1139us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:49.276349 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:19:49.276638 (+ 289us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:19:49.278140 (+ 1502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:19:49.278147 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:19:49.278179 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:19:49.278598 (+ 419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:49.278612 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.280933 (+ 2321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.280940 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:49.283591 (+ 2651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:49.283610 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.286302 (+ 2692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.286312 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:49.286577 (+ 265us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:49.287402 (+ 825us) client_negotiation.cc:769] Sending connection context
0809 12:19:49.287479 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 12:19:49.287888 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":983,"mutex_wait_us":750,"thread_start_us":835,"threads_started":1}
I20260809 12:19:49.289685 7317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.275463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59424 (local address 127.19.183.195:41617)
0809 12:19:49.277213 (+ 1750us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:49.277223 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:49.277259 (+ 36us) server_negotiation.cc:407] Connection header received
0809 12:19:49.277362 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:49.277371 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:49.277594 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:49.277790 (+ 196us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:19:49.278878 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.280732 (+ 1854us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.283821 (+ 3089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.285656 (+ 1835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.285875 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:49.286086 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:49.288862 (+ 2776us) server_negotiation.cc:299] Negotiation successful
0809 12:19:49.289214 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1576,"thread_start_us":1430,"threads_started":1}
I20260809 12:19:49.293002 6989 tablet_service.cc:1467] Processing CreateTablet for tablet b1863d80ed454c6491c1382bd9d5210c (DEFAULT_TABLE table=test-table [id=a86cb0d12c1847608f9d7e3d1ea44145]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:19:49.294037 7179 tablet_service.cc:1467] Processing CreateTablet for tablet b1863d80ed454c6491c1382bd9d5210c (DEFAULT_TABLE table=test-table [id=a86cb0d12c1847608f9d7e3d1ea44145]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:19:49.294839 6989 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1863d80ed454c6491c1382bd9d5210c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:19:49.295506 7179 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1863d80ed454c6491c1382bd9d5210c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:19:49.298807 7319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.276396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43825 (local address 127.0.0.1:54584)
0809 12:19:49.287087 (+ 10691us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:49.287110 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:19:49.287264 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:19:49.291389 (+ 4125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:19:49.291394 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:19:49.291419 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:19:49.291719 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:49.291730 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.293756 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.293762 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:49.295964 (+ 2202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:49.295980 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.298007 (+ 2027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.298014 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:49.298189 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:49.298222 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 12:19:49.298315 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 12:19:49.298444 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":10542,"thread_start_us":10363,"threads_started":1}
I20260809 12:19:49.299685 7320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.277025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54584 (local address 127.19.183.193:43825)
0809 12:19:49.290404 (+ 13379us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:49.290412 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:49.290442 (+ 30us) server_negotiation.cc:407] Connection header received
0809 12:19:49.290547 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:49.290555 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:49.290845 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:49.291107 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:19:49.291915 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.293542 (+ 1627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.296159 (+ 2617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.297502 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.297686 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:49.297859 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:49.299173 (+ 1314us) server_negotiation.cc:299] Negotiation successful
0809 12:19:49.299351 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13217,"thread_start_us":13038,"threads_started":1}
I20260809 12:19:49.303334 6775 tablet_service.cc:1467] Processing CreateTablet for tablet b1863d80ed454c6491c1382bd9d5210c (DEFAULT_TABLE table=test-table [id=a86cb0d12c1847608f9d7e3d1ea44145]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:19:49.304854 6775 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b1863d80ed454c6491c1382bd9d5210c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:19:49.306059 7084 heartbeater.cc:503] Master 127.19.183.254:44741 was elected leader, sending a full tablet report...
I20260809 12:19:49.317842 7323 tablet_bootstrap.cc:492] T b1863d80ed454c6491c1382bd9d5210c P a025ca2df11a430490a2fc19937ef150: Bootstrap starting.
I20260809 12:19:49.328236 7323 tablet_bootstrap.cc:654] T b1863d80ed454c6491c1382bd9d5210c P a025ca2df11a430490a2fc19937ef150: Neither blocks nor log segments found. Creating new log.
I20260809 12:19:49.329041 7321 tablet_bootstrap.cc:492] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2: Bootstrap starting.
I20260809 12:19:49.332123 7323 log.cc:824] T b1863d80ed454c6491c1382bd9d5210c P a025ca2df11a430490a2fc19937ef150: Log is configured to *not* fsync() on all Append() calls
I20260809 12:19:49.332691 7321 tablet_bootstrap.cc:654] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2: Neither blocks nor log segments found. Creating new log.
I20260809 12:19:49.333457 7324 tablet_bootstrap.cc:492] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b: Bootstrap starting.
I20260809 12:19:49.334290 7321 log.cc:824] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2: Log is configured to *not* fsync() on all Append() calls
I20260809 12:19:49.337381 7324 tablet_bootstrap.cc:654] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b: Neither blocks nor log segments found. Creating new log.
I20260809 12:19:49.338645 7324 log.cc:824] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b: Log is configured to *not* fsync() on all Append() calls
I20260809 12:19:49.340624 7321 tablet_bootstrap.cc:492] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2: No bootstrap required, opened a new log
I20260809 12:19:49.340864 7323 tablet_bootstrap.cc:492] T b1863d80ed454c6491c1382bd9d5210c P a025ca2df11a430490a2fc19937ef150: No bootstrap required, opened a new log
I20260809 12:19:49.340894 7321 ts_tablet_manager.cc:1397] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.005s
I20260809 12:19:49.341289 7323 ts_tablet_manager.cc:1397] T b1863d80ed454c6491c1382bd9d5210c P a025ca2df11a430490a2fc19937ef150: Time spent bootstrapping tablet: real 0.024s user 0.008s sys 0.006s
I20260809 12:19:49.350953 7321 raft_consensus.cc:348] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a025ca2df11a430490a2fc19937ef150" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 35717 } } peers { permanent_uuid: "02d460f29a094bcebb52310a6a8616c2" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41617 } } peers { permanent_uuid: "bd56c3f50103472eb84aad2600b2351b" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43825 } }
I20260809 12:19:49.351378 7321 raft_consensus.cc:374] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:19:49.351467 7321 raft_consensus.cc:729] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 02d460f29a094bcebb52310a6a8616c2, State: Initialized, Role: FOLLOWER
I20260809 12:19:49.352138 7321 consensus_queue.cc:260] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a025ca2df11a430490a2fc19937ef150" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 35717 } } peers { permanent_uuid: "02d460f29a094bcebb52310a6a8616c2" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41617 } } peers { permanent_uuid: "bd56c3f50103472eb84aad2600b2351b" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43825 } }
I20260809 12:19:49.357967 7323 raft_consensus.cc:348] T b1863d80ed454c6491c1382bd9d5210c P a025ca2df11a430490a2fc19937ef150 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a025ca2df11a430490a2fc19937ef150" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 35717 } } peers { permanent_uuid: "02d460f29a094bcebb52310a6a8616c2" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41617 } } peers { permanent_uuid: "bd56c3f50103472eb84aad2600b2351b" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43825 } }
I20260809 12:19:49.359455 7279 heartbeater.cc:503] Master 127.19.183.254:44741 was elected leader, sending a full tablet report...
I20260809 12:19:49.360177 7321 ts_tablet_manager.cc:1428] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2: Time spent starting tablet: real 0.019s user 0.014s sys 0.007s
I20260809 12:19:49.360603 7323 raft_consensus.cc:374] T b1863d80ed454c6491c1382bd9d5210c P a025ca2df11a430490a2fc19937ef150 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:19:49.360713 7323 raft_consensus.cc:729] T b1863d80ed454c6491c1382bd9d5210c P a025ca2df11a430490a2fc19937ef150 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a025ca2df11a430490a2fc19937ef150, State: Initialized, Role: FOLLOWER
I20260809 12:19:49.361292 7323 consensus_queue.cc:260] T b1863d80ed454c6491c1382bd9d5210c P a025ca2df11a430490a2fc19937ef150 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a025ca2df11a430490a2fc19937ef150" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 35717 } } peers { permanent_uuid: "02d460f29a094bcebb52310a6a8616c2" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41617 } } peers { permanent_uuid: "bd56c3f50103472eb84aad2600b2351b" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43825 } }
I20260809 12:19:49.361991 7324 tablet_bootstrap.cc:492] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b: No bootstrap required, opened a new log
I20260809 12:19:49.362317 7324 ts_tablet_manager.cc:1397] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b: Time spent bootstrapping tablet: real 0.029s user 0.019s sys 0.008s
I20260809 12:19:49.367326 7323 ts_tablet_manager.cc:1428] T b1863d80ed454c6491c1382bd9d5210c P a025ca2df11a430490a2fc19937ef150: Time spent starting tablet: real 0.026s user 0.014s sys 0.007s
I20260809 12:19:49.372370 7324 raft_consensus.cc:348] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a025ca2df11a430490a2fc19937ef150" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 35717 } } peers { permanent_uuid: "02d460f29a094bcebb52310a6a8616c2" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41617 } } peers { permanent_uuid: "bd56c3f50103472eb84aad2600b2351b" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43825 } }
I20260809 12:19:49.372723 7324 raft_consensus.cc:374] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:19:49.372838 7324 raft_consensus.cc:729] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd56c3f50103472eb84aad2600b2351b, State: Initialized, Role: FOLLOWER
I20260809 12:19:49.373436 7324 consensus_queue.cc:260] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a025ca2df11a430490a2fc19937ef150" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 35717 } } peers { permanent_uuid: "02d460f29a094bcebb52310a6a8616c2" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41617 } } peers { permanent_uuid: "bd56c3f50103472eb84aad2600b2351b" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43825 } }
I20260809 12:19:49.374534 6873 heartbeater.cc:503] Master 127.19.183.254:44741 was elected leader, sending a full tablet report...
I20260809 12:19:49.375077 7324 ts_tablet_manager.cc:1428] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b: Time spent starting tablet: real 0.013s user 0.009s sys 0.003s
W20260809 12:19:49.407756 7281 tablet.cc:2369] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:19:49.418314 7329 raft_consensus.cc:482] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:19:49.418591 7329 raft_consensus.cc:504] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a025ca2df11a430490a2fc19937ef150" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 35717 } } peers { permanent_uuid: "02d460f29a094bcebb52310a6a8616c2" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41617 } } peers { permanent_uuid: "bd56c3f50103472eb84aad2600b2351b" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43825 } }
I20260809 12:19:49.420337 7329 leader_election.cc:288] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a025ca2df11a430490a2fc19937ef150 (127.19.183.194:35717), bd56c3f50103472eb84aad2600b2351b (127.19.183.193:43825)
I20260809 12:19:49.421770 7331 raft_consensus.cc:482] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:19:49.422032 7331 raft_consensus.cc:504] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a025ca2df11a430490a2fc19937ef150" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 35717 } } peers { permanent_uuid: "02d460f29a094bcebb52310a6a8616c2" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41617 } } peers { permanent_uuid: "bd56c3f50103472eb84aad2600b2351b" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43825 } }
I20260809 12:19:49.423810 7331 leader_election.cc:288] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a025ca2df11a430490a2fc19937ef150 (127.19.183.194:35717), 02d460f29a094bcebb52310a6a8616c2 (127.19.183.195:41617)
I20260809 12:19:49.429603 7285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.420558 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:35717 (local address 127.19.183.195:53595)
0809 12:19:49.420938 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:49.420957 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:19:49.421151 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:19:49.421809 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:19:49.421816 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:19:49.421851 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:19:49.422108 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:49.422122 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.424056 (+ 1934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.424063 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:49.426195 (+ 2132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:49.426213 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.428597 (+ 2384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.428603 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:49.428927 (+ 324us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:49.428964 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 12:19:49.429108 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 12:19:49.429308 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"mutex_wait_us":45}
I20260809 12:19:49.430245 7314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.420731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:53595 (local address 127.19.183.194:35717)
0809 12:19:49.420960 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:49.420966 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:49.421304 (+ 338us) server_negotiation.cc:407] Connection header received
0809 12:19:49.421399 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:49.421406 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:49.421514 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:49.421603 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:19:49.422431 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.423857 (+ 1426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.427081 (+ 3224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.428272 (+ 1191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.428396 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:49.428468 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:49.429811 (+ 1343us) server_negotiation.cc:299] Negotiation successful
0809 12:19:49.429958 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 12:19:49.431739 7020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1863d80ed454c6491c1382bd9d5210c" candidate_uuid: "02d460f29a094bcebb52310a6a8616c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a025ca2df11a430490a2fc19937ef150" is_pre_election: true
I20260809 12:19:49.432614 7020 raft_consensus.cc:2454] T b1863d80ed454c6491c1382bd9d5210c P a025ca2df11a430490a2fc19937ef150 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 02d460f29a094bcebb52310a6a8616c2 in term 0.
I20260809 12:19:49.433373 7117 leader_election.cc:302] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02d460f29a094bcebb52310a6a8616c2, a025ca2df11a430490a2fc19937ef150; no voters:
I20260809 12:19:49.433820 7329 raft_consensus.cc:2781] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:19:49.433967 7329 raft_consensus.cc:482] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:19:49.434082 7329 raft_consensus.cc:3037] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:19:49.437592 7320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.421329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:42891 (local address 127.19.183.193:43825)
0809 12:19:49.421509 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:49.421514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:49.424627 (+ 3113us) server_negotiation.cc:407] Connection header received
0809 12:19:49.424933 (+ 306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:49.424938 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:49.425016 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:49.425085 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:19:49.426087 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.427274 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.435099 (+ 7825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.436302 (+ 1203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.436414 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:49.436488 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:49.437166 (+ 678us) server_negotiation.cc:299] Negotiation successful
0809 12:19:49.437321 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 12:19:49.437592 7329 raft_consensus.cc:504] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a025ca2df11a430490a2fc19937ef150" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 35717 } } peers { permanent_uuid: "02d460f29a094bcebb52310a6a8616c2" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41617 } } peers { permanent_uuid: "bd56c3f50103472eb84aad2600b2351b" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43825 } }
I20260809 12:19:49.438328 7329 leader_election.cc:288] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2 [CANDIDATE]: Term 1 election: Requested vote from peers a025ca2df11a430490a2fc19937ef150 (127.19.183.194:35717), bd56c3f50103472eb84aad2600b2351b (127.19.183.193:43825)
I20260809 12:19:49.438657 7333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.420911 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43825 (local address 127.19.183.195:42891)
0809 12:19:49.424531 (+ 3620us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:49.424551 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:19:49.424728 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:19:49.425332 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:19:49.425336 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:19:49.425361 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:19:49.425858 (+ 497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:49.425874 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.427470 (+ 1596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.427475 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:49.434923 (+ 7448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:49.434938 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.436705 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.436712 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:49.436932 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:49.436975 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 12:19:49.438003 (+ 1028us) client_negotiation.cc:240] Negotiation successful
0809 12:19:49.438147 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3499,"mutex_wait_us":7,"thread_start_us":3336,"threads_started":1}
I20260809 12:19:49.439005 7020 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1863d80ed454c6491c1382bd9d5210c" candidate_uuid: "02d460f29a094bcebb52310a6a8616c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a025ca2df11a430490a2fc19937ef150"
I20260809 12:19:49.439210 7020 raft_consensus.cc:3037] T b1863d80ed454c6491c1382bd9d5210c P a025ca2df11a430490a2fc19937ef150 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:19:49.440006 7335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.425209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:33043 (local address 127.19.183.194:35717)
0809 12:19:49.429265 (+ 4056us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:49.429270 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:49.429949 (+ 679us) server_negotiation.cc:407] Connection header received
0809 12:19:49.430011 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:49.430016 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:49.430098 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:49.430198 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:19:49.431165 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.433317 (+ 2152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.436819 (+ 3502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.438027 (+ 1208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.439431 (+ 1404us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:49.439501 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:49.439567 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 12:19:49.439685 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3922,"thread_start_us":3666,"threads_started":1}
I20260809 12:19:49.440068 7334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.424724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:35717 (local address 127.19.183.193:33043)
0809 12:19:49.429696 (+ 4972us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:49.429715 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:19:49.429847 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:19:49.430472 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:19:49.430477 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:19:49.430503 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:19:49.430935 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:49.430946 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.434062 (+ 3116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.434070 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:49.436638 (+ 2568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:49.436654 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.439117 (+ 2463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.439132 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:49.439413 (+ 281us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:49.439454 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 12:19:49.439535 (+ 81us) client_negotiation.cc:240] Negotiation successful
0809 12:19:49.439701 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4847,"mutex_wait_us":18,"thread_start_us":4684,"threads_started":1}
I20260809 12:19:49.440845 7019 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1863d80ed454c6491c1382bd9d5210c" candidate_uuid: "bd56c3f50103472eb84aad2600b2351b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a025ca2df11a430490a2fc19937ef150" is_pre_election: true
I20260809 12:19:49.441810 6807 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1863d80ed454c6491c1382bd9d5210c" candidate_uuid: "02d460f29a094bcebb52310a6a8616c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd56c3f50103472eb84aad2600b2351b"
I20260809 12:19:49.442260 6807 raft_consensus.cc:3037] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:19:49.441690 6808 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1863d80ed454c6491c1382bd9d5210c" candidate_uuid: "02d460f29a094bcebb52310a6a8616c2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd56c3f50103472eb84aad2600b2351b" is_pre_election: true
I20260809 12:19:49.443774 7317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.429962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:43567 (local address 127.19.183.195:41617)
0809 12:19:49.430131 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:49.430136 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:49.431814 (+ 1678us) server_negotiation.cc:407] Connection header received
0809 12:19:49.432074 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:49.432079 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:49.432151 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:49.432206 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:19:49.432875 (+ 669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.433939 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.441221 (+ 7282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.442497 (+ 1276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.442642 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:49.442706 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:49.443441 (+ 735us) server_negotiation.cc:299] Negotiation successful
0809 12:19:49.443560 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 12:19:49.444139 7020 raft_consensus.cc:2454] T b1863d80ed454c6491c1382bd9d5210c P a025ca2df11a430490a2fc19937ef150 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02d460f29a094bcebb52310a6a8616c2 in term 1.
I20260809 12:19:49.444825 7336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.430528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:41617 (local address 127.19.183.193:43567)
0809 12:19:49.431713 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:49.431732 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:19:49.431905 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:19:49.432370 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:19:49.432375 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:19:49.432399 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:19:49.432710 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:49.432721 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.434139 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.434145 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:49.440961 (+ 6816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:49.440976 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.443044 (+ 2068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.443052 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:49.443259 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:49.443290 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 12:19:49.444041 (+ 751us) client_negotiation.cc:240] Negotiation successful
0809 12:19:49.444391 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1056,"mutex_wait_us":3,"thread_start_us":893,"threads_started":1}
I20260809 12:19:49.444770 7117 leader_election.cc:302] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 02d460f29a094bcebb52310a6a8616c2, a025ca2df11a430490a2fc19937ef150; no voters:
I20260809 12:19:49.445766 7329 raft_consensus.cc:2781] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:19:49.446239 7329 raft_consensus.cc:686] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2 [term 1 LEADER]: Becoming Leader. State: Replica: 02d460f29a094bcebb52310a6a8616c2, State: Running, Role: LEADER
I20260809 12:19:49.446386 7206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b1863d80ed454c6491c1382bd9d5210c" candidate_uuid: "bd56c3f50103472eb84aad2600b2351b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "02d460f29a094bcebb52310a6a8616c2" is_pre_election: true
I20260809 12:19:49.447095 6807 raft_consensus.cc:2454] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 02d460f29a094bcebb52310a6a8616c2 in term 1.
I20260809 12:19:49.447086 7329 consensus_queue.cc:237] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a025ca2df11a430490a2fc19937ef150" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 35717 } } peers { permanent_uuid: "02d460f29a094bcebb52310a6a8616c2" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41617 } } peers { permanent_uuid: "bd56c3f50103472eb84aad2600b2351b" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43825 } }
I20260809 12:19:49.447387 6716 leader_election.cc:302] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd56c3f50103472eb84aad2600b2351b; no voters: 02d460f29a094bcebb52310a6a8616c2, a025ca2df11a430490a2fc19937ef150
I20260809 12:19:49.447791 7331 raft_consensus.cc:2726] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 12:19:49.452109 6582 catalog_manager.cc:5495] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 02d460f29a094bcebb52310a6a8616c2 (127.19.183.195). New cstate: current_term: 1 leader_uuid: "02d460f29a094bcebb52310a6a8616c2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a025ca2df11a430490a2fc19937ef150" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 35717 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "02d460f29a094bcebb52310a6a8616c2" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41617 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd56c3f50103472eb84aad2600b2351b" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43825 } health_report { overall_health: UNKNOWN } } }
I20260809 12:19:49.476441 7317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.471026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59440 (local address 127.19.183.195:41617)
0809 12:19:49.471224 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:49.471230 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:49.471250 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:19:49.471379 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:49.471385 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:49.471474 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:49.471620 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:19:49.472023 (+ 403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.472769 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.474141 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.474533 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.474662 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:49.475203 (+ 541us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:19:49.475859 (+ 656us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:19:49.476004 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:49.476082 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 12:19:49.476168 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 12:19:49.485210 7020 raft_consensus.cc:1261] T b1863d80ed454c6491c1382bd9d5210c P a025ca2df11a430490a2fc19937ef150 [term 1 FOLLOWER]: Refusing update from remote peer 02d460f29a094bcebb52310a6a8616c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:19:49.485330 6807 raft_consensus.cc:1261] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b [term 1 FOLLOWER]: Refusing update from remote peer 02d460f29a094bcebb52310a6a8616c2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:19:49.486102 7337 consensus_queue.cc:1035] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a025ca2df11a430490a2fc19937ef150" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 35717 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:19:49.486475 7329 consensus_queue.cc:1035] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd56c3f50103472eb84aad2600b2351b" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:19:49.535565 6582 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:19:49.536100 6582 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:19:49.545888 6582 catalog_manager.cc:5782] T 00000000000000000000000000000000 P fd9e554e724e4b5a9d881b1659bb5def: Sending DeleteTablet for 3 replicas of tablet b1863d80ed454c6491c1382bd9d5210c
I20260809 12:19:49.547797 6775 tablet_service.cc:1514] Processing DeleteTablet for tablet b1863d80ed454c6491c1382bd9d5210c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:19:49 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:54584
I20260809 12:19:49.549811 6989 tablet_service.cc:1514] Processing DeleteTablet for tablet b1863d80ed454c6491c1382bd9d5210c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:19:49 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:45908
I20260809 12:19:49.551936 7179 tablet_service.cc:1514] Processing DeleteTablet for tablet b1863d80ed454c6491c1382bd9d5210c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:19:49 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:59424
I20260809 12:19:49.552842 7353 tablet_replica.cc:330] T b1863d80ed454c6491c1382bd9d5210c P a025ca2df11a430490a2fc19937ef150: stopping tablet replica
I20260809 12:19:49.553444 7353 raft_consensus.cc:2229] T b1863d80ed454c6491c1382bd9d5210c P a025ca2df11a430490a2fc19937ef150 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:19:49.553418 6774 tablet_service.cc:1514] Processing DeleteTablet for tablet b1863d80ed454c6491c1382bd9d5210c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:19:49 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:54584
I20260809 12:19:49.554462 7353 raft_consensus.cc:2258] T b1863d80ed454c6491c1382bd9d5210c P a025ca2df11a430490a2fc19937ef150 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:19:49.556226 7352 tablet_replica.cc:330] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b: stopping tablet replica
I20260809 12:19:49.556808 7352 raft_consensus.cc:2229] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:19:49.557760 7352 raft_consensus.cc:2258] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:19:49.560747 7353 ts_tablet_manager.cc:1905] T b1863d80ed454c6491c1382bd9d5210c P a025ca2df11a430490a2fc19937ef150: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:19:49.561192 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 6630
I20260809 12:19:49.570865 7353 ts_tablet_manager.cc:1918] T b1863d80ed454c6491c1382bd9d5210c P a025ca2df11a430490a2fc19937ef150: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:19:49.571189 7353 log.cc:1196] T b1863d80ed454c6491c1382bd9d5210c P a025ca2df11a430490a2fc19937ef150: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/b1863d80ed454c6491c1382bd9d5210c
I20260809 12:19:49.572219 7353 ts_tablet_manager.cc:1939] T b1863d80ed454c6491c1382bd9d5210c P a025ca2df11a430490a2fc19937ef150: Deleting consensus metadata
I20260809 12:19:49.574608 6563 catalog_manager.cc:4841] TS a025ca2df11a430490a2fc19937ef150 (127.19.183.194:35717): tablet b1863d80ed454c6491c1382bd9d5210c (table test-table [id=a86cb0d12c1847608f9d7e3d1ea44145]) successfully deleted
I20260809 12:19:49.576953 7354 tablet_replica.cc:330] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2: stopping tablet replica
I20260809 12:19:49.577507 7354 raft_consensus.cc:2229] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:19:49.577967 7352 ts_tablet_manager.cc:1905] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:19:49.578454 7354 raft_consensus.cc:2258] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:19:49.593209 7352 ts_tablet_manager.cc:1918] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:19:49.593501 7352 log.cc:1196] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/b1863d80ed454c6491c1382bd9d5210c
I20260809 12:19:49.595780 7320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.563258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54590 (local address 127.19.183.193:43825)
0809 12:19:49.563404 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:49.563410 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:49.563427 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:19:49.563481 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:49.563509 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:49.563578 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:49.563694 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:49.566095 (+ 2401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.567647 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.572504 (+ 4857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.573638 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.573801 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:49.585589 (+ 11788us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:49.585604 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:49.585623 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:49.585718 (+ 95us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:49.591270 (+ 5552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:49.592386 (+ 1116us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:49.592398 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:49.592405 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:49.592575 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:49.593080 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:49.593087 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:49.593092 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:49.593972 (+ 880us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:49.594473 (+ 501us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:49.594766 (+ 293us) server_negotiation.cc:299] Negotiation successful
0809 12:19:49.595079 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 12:19:49.596606 6868 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:19:49.602960 7354 ts_tablet_manager.cc:1905] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:19:49.615465 7354 ts_tablet_manager.cc:1918] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:19:49.615801 7354 log.cc:1196] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/b1863d80ed454c6491c1382bd9d5210c
I20260809 12:19:49.616874 7354 ts_tablet_manager.cc:1939] T b1863d80ed454c6491c1382bd9d5210c P 02d460f29a094bcebb52310a6a8616c2: Deleting consensus metadata
I20260809 12:19:49.619210 6563 catalog_manager.cc:4841] TS 02d460f29a094bcebb52310a6a8616c2 (127.19.183.195:41617): tablet b1863d80ed454c6491c1382bd9d5210c (table test-table [id=a86cb0d12c1847608f9d7e3d1ea44145]) successfully deleted
I20260809 12:19:49.734174 7352 ts_tablet_manager.cc:1939] T b1863d80ed454c6491c1382bd9d5210c P bd56c3f50103472eb84aad2600b2351b: Deleting consensus metadata
I20260809 12:19:49.735095 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 6630
W20260809 12:19:49.769243 6563 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.183.193:43825 (error 108)
I20260809 12:19:49.769627 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 6877
I20260809 12:19:49.771793 7319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.771308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43825 (local address 127.0.0.1:54600)
0809 12:19:49.771501 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:49.771630 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260809 12:19:49.772089 6563 catalog_manager.cc:4584] TS bd56c3f50103472eb84aad2600b2351b (127.19.183.193:43825): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b1863d80ed454c6491c1382bd9d5210c: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43825: connect: Connection refused (error 111)
I20260809 12:19:49.785077 7335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.770986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45914 (local address 127.19.183.194:35717)
0809 12:19:49.771234 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:49.771243 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:49.771261 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:19:49.771328 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:49.771336 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:49.771416 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:49.771517 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:49.773537 (+ 2020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.774270 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.775798 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.776215 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.776358 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:49.778908 (+ 2550us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:49.778920 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:49.778934 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:49.778968 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:49.781670 (+ 2702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:49.782409 (+ 739us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:49.782417 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:49.782420 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:49.782677 (+ 257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:49.783136 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:49.783139 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:49.783141 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:49.783550 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:49.784180 (+ 630us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:49.784339 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 12:19:49.784649 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 12:19:49.785823 7077 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:19:49.816740 7319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.816198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43825 (local address 127.0.0.1:54606)
0809 12:19:49.816417 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:49.816565 (+ 148us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
I20260809 12:19:49.817833 7319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.817481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43825 (local address 127.0.0.1:54612)
0809 12:19:49.817646 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:49.817712 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 12:19:49.825968 7319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.825580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43825 (local address 127.0.0.1:54614)
0809 12:19:49.825764 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:49.825835 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 12:19:49.826952 7319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.826593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43825 (local address 127.0.0.1:54628)
0809 12:19:49.826777 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:49.826848 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 12:19:49.866387 7319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.865929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43825 (local address 127.0.0.1:54636)
0809 12:19:49.866127 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:49.866215 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 12:19:49.867499 7319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.867125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43825 (local address 127.0.0.1:54650)
0809 12:19:49.867314 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:49.867398 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260809 12:19:49.904107 7319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.903608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43825 (local address 127.0.0.1:54660)
0809 12:19:49.903831 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:49.903930 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260809 12:19:49.905187 7319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.904851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43825 (local address 127.0.0.1:54666)
0809 12:19:49.905003 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:49.905074 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 12:19:49.935171 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 6877
I20260809 12:19:49.959926 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 7090
I20260809 12:19:49.968940 7319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.968356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43825 (local address 127.0.0.1:54670)
0809 12:19:49.968617 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:49.968724 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260809 12:19:49.970225 7319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.969778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43825 (local address 127.0.0.1:54678)
0809 12:19:49.969976 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:49.970070 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 12:19:49.975633 7317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:49.960806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59442 (local address 127.19.183.195:41617)
0809 12:19:49.960988 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:49.960996 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:49.961050 (+ 54us) server_negotiation.cc:407] Connection header received
0809 12:19:49.961305 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:49.961311 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:49.961400 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:49.961515 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:49.962965 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.963707 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.965334 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:49.965789 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:49.965954 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:49.969074 (+ 3120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:49.969095 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:49.969117 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:49.969163 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:49.972709 (+ 3546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:49.973433 (+ 724us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:49.973442 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:49.973444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:49.973579 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:49.973978 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:49.973985 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:49.973992 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:49.974391 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:49.974786 (+ 395us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:49.975082 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 12:19:49.975315 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 12:19:49.976316 7272 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:19:50.007076 7319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:50.006450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43825 (local address 127.0.0.1:54688)
0809 12:19:50.006743 (+ 293us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:50.006871 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260809 12:19:50.008189 7319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:50.007818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43825 (local address 127.0.0.1:54702)
0809 12:19:50.007973 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:50.008059 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 12:19:50.105090 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 7090
I20260809 12:19:50.133983 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 6540
I20260809 12:19:50.135254 6620 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:19:50.249375 7319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:50.248326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43825 (local address 127.0.0.1:54704)
0809 12:19:50.249000 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:50.249154 (+ 154us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":530}
I20260809 12:19:50.249830 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 6540
I20260809 12:19:50.283875 20191 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 12:19:50.628998 20191 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 12:19:50.629228 20191 mini_ranger.cc:66] Stopping Ranger...
I20260809 12:19:50.985375 20191 mini_ranger.cc:68] Stopped Ranger
2026-08-09 12:19:50.985 UTC [5137] LOG: received smart shutdown request
2026-08-09 12:19:50.988 UTC [5137] LOG: background worker "logical replication launcher" (PID 5154) exited with exit code 1
2026-08-09 12:19:50.988 UTC [5144] LOG: shutting down
2026-08-09 12:19:51.137 UTC [5137] LOG: database system is shut down
2026-08-09T12:19:51Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (53691 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:19:51 dist-test-slave-39l8 krb5kdc[7373](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:19:51 dist-test-slave-39l8 krb5kdc[7373](info): set up 2 sockets
Aug 09 12:19:51 dist-test-slave-39l8 krb5kdc[7373](info): commencing operation
krb5kdc: starting...
W20260809 12:19:52.580175 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.273s user 0.000s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:19:52 dist-test-slave-39l8 krb5kdc[7373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:19:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:19:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:19:52.872428 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:33259
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:44791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:33259,127.19.183.253:41169,127.19.183.252:34787
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:19:53.217679 7396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:19:53.217973 7396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:19:53.218073 7396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:19:53.225287 7396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:19:53.225400 7396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:19:53.225456 7396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:19:53.225515 7396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:19:53.225560 7396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:19:53.235327 7396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:44791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:33259,127.19.183.253:41169,127.19.183.252:34787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:33259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:19:53.236630 7396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:19:53.237869 7396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:19:53.250094 7406 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:19:53.250551 7409 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:19:53.250635 7403 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:19:53.251078 7396 server_base.cc:1034] running on GCE node
I20260809 12:19:53.251927 7396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:19:53.253314 7396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:19:53.254575 7396 hybrid_clock.cc:648] HybridClock initialized: now 1786277993254530 us; error 56 us; skew 500 ppm
I20260809 12:19:53.254974 7396 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:19:53 dist-test-slave-39l8 krb5kdc[7373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277993, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:19:53.263203 7396 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:19:53.264765 7396 webserver.cc:460] Webserver started at http://127.19.183.254:43075/ using document root <none> and password file <none>
I20260809 12:19:53.265504 7396 fs_manager.cc:359] Metadata directory not provided
I20260809 12:19:53.265659 7396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:19:53.266008 7396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:19:53.269169 7396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "47bd69cd65694cb6a00995695e1c1a68"
format_stamp: "Formatted at 2026-08-09 12:19:53 on dist-test-slave-39l8"
server_key: "75d115189c0963a15d55ac8d5a85e4de"
server_key_iv: "393c14159758ae1a71692a41267fcd9b"
server_key_version: "encryptionkey@0"
I20260809 12:19:53.270030 7396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "47bd69cd65694cb6a00995695e1c1a68"
format_stamp: "Formatted at 2026-08-09 12:19:53 on dist-test-slave-39l8"
server_key: "75d115189c0963a15d55ac8d5a85e4de"
server_key_iv: "393c14159758ae1a71692a41267fcd9b"
server_key_version: "encryptionkey@0"
I20260809 12:19:53.275612 7396 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 12:19:53.279028 7421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:19:53.280326 7396 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:19:53.280524 7396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "47bd69cd65694cb6a00995695e1c1a68"
format_stamp: "Formatted at 2026-08-09 12:19:53 on dist-test-slave-39l8"
server_key: "75d115189c0963a15d55ac8d5a85e4de"
server_key_iv: "393c14159758ae1a71692a41267fcd9b"
server_key_version: "encryptionkey@0"
I20260809 12:19:53.280730 7396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:19:53.294281 7396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:19:53.298276 7396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:19:53.298588 7396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:19:53.317562 7396 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:33259
I20260809 12:19:53.317585 7490 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:33259 every 8 connection(s)
I20260809 12:19:53.319075 7396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:19:53.321062 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 7396
I20260809 12:19:53.321321 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:19:53.322042 20191 external_mini_cluster.cc:1428] Setting key 5ffb3f32b623498b777f86a770afcef4
I20260809 12:19:53.322892 7495 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:19:53.329293 7495 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33259 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:19:53 dist-test-slave-39l8 krb5kdc[7373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277993, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.183.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.253@KRBTEST.COM" created.
I20260809 12:19:53.354205 7499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:53.333831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52494 (local address 127.19.183.254:33259)
0809 12:19:53.334528 (+ 697us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:53.334533 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:53.334583 (+ 50us) server_negotiation.cc:407] Connection header received
0809 12:19:53.336856 (+ 2273us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:53.336861 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:53.337073 (+ 212us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:53.337236 (+ 163us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:53.338992 (+ 1756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:53.340441 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:53.342080 (+ 1639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:53.342614 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:53.342869 (+ 255us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:53.346605 (+ 3736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:53.346614 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:53.346626 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:53.346659 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:53.349577 (+ 2918us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:53.350303 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:53.350318 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:53.350321 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:53.350466 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:53.350939 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:53.350945 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:53.350951 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:53.351486 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:53.351715 (+ 229us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:53.352326 (+ 611us) server_negotiation.cc:299] Negotiation successful
0809 12:19:53.352678 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":36,"server-negotiator.queue_time_us":504,"thread_start_us":342,"threads_started":1}
I20260809 12:19:53.354228 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:53.332544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:33259 (local address 127.0.0.1:52494)
0809 12:19:53.333396 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:53.333436 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 12:19:53.336482 (+ 3046us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:19:53.337326 (+ 844us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:19:53.337336 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:19:53.338232 (+ 896us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:19:53.338774 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:53.338802 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:53.340609 (+ 1807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:53.340615 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:53.341923 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:53.341937 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:53.342894 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:53.342900 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:53.343089 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:53.343814 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:19:53.343840 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:19:53.346375 (+ 2535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:19:53.349717 (+ 3342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:53.349730 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:53.349754 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:53.350170 (+ 416us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:53.350591 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:53.350596 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:53.350600 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:53.350831 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:53.351629 (+ 798us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:19:53.351638 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:19:53.351867 (+ 229us) client_negotiation.cc:769] Sending connection context
0809 12:19:53.352182 (+ 315us) client_negotiation.cc:240] Negotiation successful
0809 12:19:53.352791 (+ 609us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"mutex_wait_us":157,"thread_start_us":399,"threads_started":1}
I20260809 12:19:53.357636 7495 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:19:53.358904 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:53.358374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:41169 (local address 127.0.0.1:40928)
0809 12:19:53.358596 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:53.358756 (+ 160us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:41169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260809 12:19:53.359294 7425 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.183.253:41169: connect: Connection refused (error 111)
I20260809 12:19:53.361336 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:53.360950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:41169 (local address 127.0.0.1:40932)
0809 12:19:53.361134 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:53.361201 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:41169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 12:19:53.361550 7495 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.253:41169: Network error: Client connection negotiation failed: client connection to 127.19.183.253:41169: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.253.keytab.
Entry for principal kudu/127.19.183.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.253.keytab.
I20260809 12:19:53.391847 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.253:41169
--webserver_interface=127.19.183.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.253.keytab
--principal=kudu/127.19.183.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:44791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:33259,127.19.183.253:41169,127.19.183.252:34787
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:19:53.410959 7495 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } attempt: 1
I20260809 12:19:53.412192 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:53.411686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:41169 (local address 127.0.0.1:40938)
0809 12:19:53.411906 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:53.412058 (+ 152us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:41169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 12:19:53.413369 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:53.412987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:41169 (local address 127.0.0.1:40952)
0809 12:19:53.413179 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:53.413244 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:41169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260809 12:19:53.413581 7495 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.253:41169: Network error: Client connection negotiation failed: client connection to 127.19.183.253:41169: connect: Connection refused (error 111)
I20260809 12:19:53.481833 7495 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } attempt: 2
I20260809 12:19:53.483194 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:53.482672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:41169 (local address 127.0.0.1:40960)
0809 12:19:53.482947 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:53.483049 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:41169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260809 12:19:53.484469 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:53.484036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:41169 (local address 127.0.0.1:40966)
0809 12:19:53.484262 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:53.484341 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:41169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260809 12:19:53.484853 7495 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.253:41169: Network error: Client connection negotiation failed: client connection to 127.19.183.253:41169: connect: Connection refused (error 111)
I20260809 12:19:53.576117 7495 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } attempt: 3
I20260809 12:19:53.577385 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:53.576923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:41169 (local address 127.0.0.1:40976)
0809 12:19:53.577140 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:53.577250 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:41169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 12:19:53.578518 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:53.578174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:41169 (local address 127.0.0.1:40992)
0809 12:19:53.578314 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:53.578384 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:41169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 12:19:53.578831 7495 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.253:41169: Network error: Client connection negotiation failed: client connection to 127.19.183.253:41169: connect: Connection refused (error 111)
I20260809 12:19:53.722103 7495 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } attempt: 4
I20260809 12:19:53.723491 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:53.722936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:41169 (local address 127.0.0.1:41008)
0809 12:19:53.723261 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:53.723350 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:41169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":179}
I20260809 12:19:53.724632 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:53.724183 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:41169 (local address 127.0.0.1:41024)
0809 12:19:53.724354 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:53.724469 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:41169: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260809 12:19:53.724823 7495 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.253:41169: Network error: Client connection negotiation failed: client connection to 127.19.183.253:41169: connect: Connection refused (error 111)
W20260809 12:19:53.735879 7503 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:19:53.736186 7503 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:19:53.736354 7503 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:19:53.743724 7503 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:19:53.743873 7503 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:19:53.743968 7503 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:19:53.744077 7503 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:19:53.744141 7503 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:19:53.754014 7503 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:44791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:33259,127.19.183.253:41169,127.19.183.252:34787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.253.keytab
--principal=kudu/127.19.183.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.253:41169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.183.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:19:53.755393 7503 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:19:53.756739 7503 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:19:53.768371 7513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:19:53.768462 7515 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:19:53.768419 7519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:19:53.769865 7503 server_base.cc:1034] running on GCE node
I20260809 12:19:53.770780 7503 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:19:53.772202 7503 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:19:53.773442 7503 hybrid_clock.cc:648] HybridClock initialized: now 1786277993773400 us; error 55 us; skew 500 ppm
I20260809 12:19:53.773797 7503 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:19:53 dist-test-slave-39l8 krb5kdc[7373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277993, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:19:53.776995 7503 init.cc:377] Logged in from keytab as kudu/127.19.183.253@KRBTEST.COM (short username kudu)
I20260809 12:19:53.778664 7503 webserver.cc:460] Webserver started at http://127.19.183.253:34003/ using document root <none> and password file <none>
I20260809 12:19:53.779505 7503 fs_manager.cc:359] Metadata directory not provided
I20260809 12:19:53.779609 7503 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:19:53.779881 7503 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:19:53.782981 7503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-1/data/instance:
uuid: "c7df1a80f5b84eb6976d78f2a387ebc1"
format_stamp: "Formatted at 2026-08-09 12:19:53 on dist-test-slave-39l8"
server_key: "7178da7bf03fb39a16e0345e8ecb68b0"
server_key_iv: "c2dae257c1ea5f910d96075ee26d61bf"
server_key_version: "encryptionkey@0"
I20260809 12:19:53.783773 7503 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-1/wal/instance:
uuid: "c7df1a80f5b84eb6976d78f2a387ebc1"
format_stamp: "Formatted at 2026-08-09 12:19:53 on dist-test-slave-39l8"
server_key: "7178da7bf03fb39a16e0345e8ecb68b0"
server_key_iv: "c2dae257c1ea5f910d96075ee26d61bf"
server_key_version: "encryptionkey@0"
I20260809 12:19:53.789321 7503 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:19:53.792950 7530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:19:53.794313 7503 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:19:53.794494 7503 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-1/wal
uuid: "c7df1a80f5b84eb6976d78f2a387ebc1"
format_stamp: "Formatted at 2026-08-09 12:19:53 on dist-test-slave-39l8"
server_key: "7178da7bf03fb39a16e0345e8ecb68b0"
server_key_iv: "c2dae257c1ea5f910d96075ee26d61bf"
server_key_version: "encryptionkey@0"
I20260809 12:19:53.794713 7503 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:19:53.810446 7503 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:19:53.814498 7503 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:19:53.814824 7503 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:19:53.834635 7503 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.253:41169
I20260809 12:19:53.834677 7599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.253:41169 every 8 connection(s)
I20260809 12:19:53.836169 7503 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-1/data/info.pb
I20260809 12:19:53.839927 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 7503
I20260809 12:19:53.840126 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-1/wal/instance
I20260809 12:19:53.840266 7600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:19:53.840752 20191 external_mini_cluster.cc:1428] Setting key 5b52f051da1599b03cca1e74a4e1429a
I20260809 12:19:53.850687 7600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33259 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:19:53 dist-test-slave-39l8 krb5kdc[7373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277993, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.253@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
WARNING: no policy specified for kudu/127.19.183.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.252@KRBTEST.COM" created.
I20260809 12:19:53.873224 7499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:53.854466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52502 (local address 127.19.183.254:33259)
0809 12:19:53.854729 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:53.854737 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:53.856114 (+ 1377us) server_negotiation.cc:407] Connection header received
0809 12:19:53.858107 (+ 1993us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:53.858114 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:53.858211 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:53.858335 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:53.859970 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:53.860698 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:53.862267 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:53.862741 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:53.862889 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:53.867044 (+ 4155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:53.867049 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:53.867052 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:53.867084 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:53.869872 (+ 2788us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:53.870899 (+ 1027us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:53.870904 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:53.870906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:53.871111 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:53.871520 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:53.871523 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:53.871525 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:53.871814 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:53.871931 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:53.872696 (+ 765us) server_negotiation.cc:299] Negotiation successful
0809 12:19:53.872920 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 12:19:53.874302 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:53.855101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:33259 (local address 127.0.0.1:52502)
0809 12:19:53.855914 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:53.855962 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 12:19:53.857795 (+ 1833us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:19:53.858600 (+ 805us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:19:53.858609 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:19:53.859200 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:19:53.859778 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:53.859796 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:53.860871 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:53.860875 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:53.862106 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:53.862124 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:53.862996 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:53.863003 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:53.863222 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:53.863940 (+ 718us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:19:53.863964 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:19:53.866807 (+ 2843us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:19:53.870053 (+ 3246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:53.870066 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:53.870088 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:53.870774 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:53.871228 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:53.871232 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:53.871234 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:53.871413 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:53.871926 (+ 513us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:19:53.871932 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:19:53.872283 (+ 351us) client_negotiation.cc:769] Sending connection context
0809 12:19:53.872509 (+ 226us) client_negotiation.cc:240] Negotiation successful
0809 12:19:53.873010 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":375,"threads_started":1}
I20260809 12:19:53.876298 7600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:19:53 dist-test-slave-39l8 krb5kdc[7373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277993, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.253@KRBTEST.COM for kudu/127.19.183.253@KRBTEST.COM
I20260809 12:19:53.890996 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:53.877005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:41169 (local address 127.0.0.1:41032)
0809 12:19:53.877204 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:53.877220 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:19:53.877305 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:19:53.878337 (+ 1032us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:19:53.878341 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:19:53.878835 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:19:53.878966 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:53.878974 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:53.880336 (+ 1362us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:53.880343 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:53.881168 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:53.881180 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:53.882021 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:53.882027 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:53.882236 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:53.882859 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:19:53.882876 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:19:53.885007 (+ 2131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:19:53.888401 (+ 3394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:53.888406 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:53.888410 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:53.888768 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:53.889146 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:53.889151 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:53.889155 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:53.889300 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:53.889897 (+ 597us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:19:53.889904 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:19:53.890063 (+ 159us) client_negotiation.cc:769] Sending connection context
0809 12:19:53.890212 (+ 149us) client_negotiation.cc:240] Negotiation successful
0809 12:19:53.890498 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260809 12:19:53.891316 7606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:53.877296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41032 (local address 127.19.183.253:41169)
0809 12:19:53.877939 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:53.877944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:53.877968 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:19:53.878011 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:53.878017 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:53.878189 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:53.878341 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:53.879137 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:53.880185 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:53.881316 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:53.881852 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:53.881986 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:53.885220 (+ 3234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:53.885232 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:53.885253 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:53.885311 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:53.888242 (+ 2931us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:53.888897 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:53.888906 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:53.888911 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:53.889029 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:53.889404 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:53.889411 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:53.889414 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:53.889769 (+ 355us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:53.889995 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:53.890353 (+ 358us) server_negotiation.cc:299] Negotiation successful
0809 12:19:53.890664 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":497,"thread_start_us":361,"threads_started":1}
I20260809 12:19:53.893633 7600 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:19:53.895056 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:53.894447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:34787 (local address 127.0.0.1:50752)
0809 12:19:53.894677 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:53.894848 (+ 171us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 12:19:53.895396 7531 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
I20260809 12:19:53.897145 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:53.896678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:34787 (local address 127.0.0.1:50756)
0809 12:19:53.896875 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:53.896990 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260809 12:19:53.897384 7600 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:34787: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.252.keytab.
Entry for principal kudu/127.19.183.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.252.keytab.
I20260809 12:19:53.908147 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.252:34787
--webserver_interface=127.19.183.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.252.keytab
--principal=kudu/127.19.183.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:44791
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:33259,127.19.183.253:41169,127.19.183.252:34787
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:19:53.946759 7600 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } attempt: 1
I20260809 12:19:53.948128 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:53.947576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:34787 (local address 127.0.0.1:50762)
0809 12:19:53.947804 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:53.947937 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 12:19:53.949182 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:53.948808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:34787 (local address 127.0.0.1:50770)
0809 12:19:53.948975 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:53.949057 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 12:19:53.949406 7600 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:34787: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
I20260809 12:19:54.017653 7600 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } attempt: 2
I20260809 12:19:54.019158 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.018428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:34787 (local address 127.0.0.1:50780)
0809 12:19:54.018796 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:54.018908 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":193}
I20260809 12:19:54.020550 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.020048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:34787 (local address 127.0.0.1:50790)
0809 12:19:54.020285 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:54.020375 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260809 12:19:54.020808 7600 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:34787: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
I20260809 12:19:54.024139 7495 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } attempt: 5
Aug 09 12:19:54 dist-test-slave-39l8 krb5kdc[7373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277993, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for kudu/127.19.183.253@KRBTEST.COM
I20260809 12:19:54.037379 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.024791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:41169 (local address 127.0.0.1:41038)
0809 12:19:54.024966 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:54.024985 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:19:54.025131 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:19:54.025505 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:19:54.025513 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:19:54.025920 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:19:54.026103 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:54.026113 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.027202 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.027209 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:54.028026 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:54.028046 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.028811 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.028816 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:54.029017 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:54.029649 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:19:54.029666 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:19:54.031777 (+ 2111us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:19:54.034851 (+ 3074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:54.034857 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:54.034861 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:54.035251 (+ 390us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:54.035664 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:54.035669 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:54.035673 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:54.035815 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:54.036364 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:19:54.036369 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:19:54.036523 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 12:19:54.036677 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 12:19:54.037004 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 12:19:54.037374 7606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.024873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41038 (local address 127.19.183.253:41169)
0809 12:19:54.025073 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:54.025079 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:54.025101 (+ 22us) server_negotiation.cc:407] Connection header received
0809 12:19:54.025283 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:54.025288 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:54.025359 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:54.025462 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:54.026290 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.027059 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.028194 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.028661 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.028795 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:54.031965 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:54.031970 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:54.031973 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:54.032013 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:54.034649 (+ 2636us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.035387 (+ 738us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.035393 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.035397 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.035540 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.035929 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.035934 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.035947 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.036236 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:54.036397 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:54.036809 (+ 412us) server_negotiation.cc:299] Negotiation successful
0809 12:19:54.037027 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 12:19:54.038556 7495 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:19:54.039674 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.039293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:34787 (local address 127.0.0.1:50806)
0809 12:19:54.039476 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:54.039549 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 12:19:54.040673 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.040346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:34787 (local address 127.0.0.1:50816)
0809 12:19:54.040505 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:54.040566 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 12:19:54.040889 7495 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:34787: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
I20260809 12:19:54.092192 7495 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } attempt: 1
I20260809 12:19:54.093489 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.093007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:34787 (local address 127.0.0.1:50824)
0809 12:19:54.093218 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:54.093331 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 12:19:54.094548 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.094249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:34787 (local address 127.0.0.1:50838)
0809 12:19:54.094377 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:54.094447 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 12:19:54.094826 7495 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:34787: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
I20260809 12:19:54.112069 7600 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } attempt: 3
I20260809 12:19:54.113376 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.112828 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:34787 (local address 127.0.0.1:50844)
0809 12:19:54.113094 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:54.113221 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":100}
I20260809 12:19:54.114481 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.114047 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:34787 (local address 127.0.0.1:50852)
0809 12:19:54.114286 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:54.114364 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
W20260809 12:19:54.114717 7600 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:34787: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
I20260809 12:19:54.163069 7495 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } attempt: 2
I20260809 12:19:54.164383 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.163819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:34787 (local address 127.0.0.1:50866)
0809 12:19:54.164111 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:54.164213 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
I20260809 12:19:54.165408 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.165067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:34787 (local address 127.0.0.1:50880)
0809 12:19:54.165224 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:54.165308 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 12:19:54.165644 7495 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:34787: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
I20260809 12:19:54.258028 7600 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } attempt: 4
I20260809 12:19:54.259238 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.258757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:34787 (local address 127.0.0.1:50884)
0809 12:19:54.258992 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:54.259077 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 12:19:54.260267 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.259926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:34787 (local address 127.0.0.1:50888)
0809 12:19:54.260104 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:54.260172 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 12:19:54.260474 7600 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:34787: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
W20260809 12:19:54.262805 7610 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:19:54.263124 7610 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:19:54.263279 7610 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:19:54.270601 7610 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:19:54.270736 7610 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:19:54.270843 7610 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:19:54.270937 7610 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:19:54.271018 7610 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:19:54.271857 7495 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } attempt: 3
I20260809 12:19:54.272886 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.272438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:34787 (local address 127.0.0.1:50892)
0809 12:19:54.272642 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:54.272733 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 12:19:54.273891 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.273536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:34787 (local address 127.0.0.1:50902)
0809 12:19:54.273716 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:54.273801 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
W20260809 12:19:54.274115 7495 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:34787: Network error: Client connection negotiation failed: client connection to 127.19.183.252:34787: connect: Connection refused (error 111)
I20260809 12:19:54.281250 7610 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:44791
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:33259,127.19.183.253:41169,127.19.183.252:34787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.252.keytab
--principal=kudu/127.19.183.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.252:34787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.183.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:19:54.282613 7610 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:19:54.283911 7610 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:19:54.296358 7619 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:19:54.296278 7621 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:19:54.296849 7618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:19:54.297353 7610 server_base.cc:1034] running on GCE node
I20260809 12:19:54.298105 7610 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:19:54.299561 7610 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:19:54.300817 7610 hybrid_clock.cc:648] HybridClock initialized: now 1786277994300728 us; error 75 us; skew 500 ppm
I20260809 12:19:54.301222 7610 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:19:54 dist-test-slave-39l8 krb5kdc[7373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277994, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:19:54.304517 7610 init.cc:377] Logged in from keytab as kudu/127.19.183.252@KRBTEST.COM (short username kudu)
I20260809 12:19:54.306025 7610 webserver.cc:460] Webserver started at http://127.19.183.252:41805/ using document root <none> and password file <none>
I20260809 12:19:54.306805 7610 fs_manager.cc:359] Metadata directory not provided
I20260809 12:19:54.306952 7610 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:19:54.307284 7610 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:19:54.310205 7610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-2/data/instance:
uuid: "0ea541d65a024698ad6dce134e03c7a7"
format_stamp: "Formatted at 2026-08-09 12:19:54 on dist-test-slave-39l8"
server_key: "06d519ba51fb3ae65ce97370f8aef40a"
server_key_iv: "fe6eec61db0e23c23c6d998ae13f8dc4"
server_key_version: "encryptionkey@0"
I20260809 12:19:54.311022 7610 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-2/wal/instance:
uuid: "0ea541d65a024698ad6dce134e03c7a7"
format_stamp: "Formatted at 2026-08-09 12:19:54 on dist-test-slave-39l8"
server_key: "06d519ba51fb3ae65ce97370f8aef40a"
server_key_iv: "fe6eec61db0e23c23c6d998ae13f8dc4"
server_key_version: "encryptionkey@0"
I20260809 12:19:54.316112 7610 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 12:19:54.319726 7634 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:19:54.321071 7610 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:19:54.321251 7610 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-2/wal
uuid: "0ea541d65a024698ad6dce134e03c7a7"
format_stamp: "Formatted at 2026-08-09 12:19:54 on dist-test-slave-39l8"
server_key: "06d519ba51fb3ae65ce97370f8aef40a"
server_key_iv: "fe6eec61db0e23c23c6d998ae13f8dc4"
server_key_version: "encryptionkey@0"
I20260809 12:19:54.321434 7610 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:19:54.351622 7610 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:19:54.356088 7610 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:19:54.356428 7610 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:19:54.375407 7610 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.252:34787
I20260809 12:19:54.375447 7719 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.252:34787 every 8 connection(s)
I20260809 12:19:54.376737 7610 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-2/data/info.pb
I20260809 12:19:54.377971 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 7610
I20260809 12:19:54.378222 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786277774779245-20191-0/minicluster-data/master-2/wal/instance
I20260809 12:19:54.378854 20191 external_mini_cluster.cc:1428] Setting key 2cff33907bd110cc76c3595ad284de20
I20260809 12:19:54.381449 7720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:19:54.389802 7720 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33259 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:19:54 dist-test-slave-39l8 krb5kdc[7373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.252@KRBTEST.COM
Aug 09 12:19:54 dist-test-slave-39l8 krb5kdc[7373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277994, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.252@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:19:54.404326 7724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.380954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50912 (local address 127.19.183.252:34787)
0809 12:19:54.381802 (+ 848us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:54.381813 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:54.381860 (+ 47us) server_negotiation.cc:407] Connection header received
0809 12:19:54.383587 (+ 1727us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:54.383610 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:54.383995 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:54.384414 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:54.386295 (+ 1881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.387702 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.389545 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.390215 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.390359 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:54.395374 (+ 5015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:54.395381 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:54.395391 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:54.395430 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:54.398738 (+ 3308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.399709 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.399718 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.399756 (+ 38us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.400109 (+ 353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.400639 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.400644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.400647 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.401535 (+ 888us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:54.401847 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:54.402406 (+ 559us) server_negotiation.cc:299] Negotiation successful
0809 12:19:54.402882 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":640,"thread_start_us":401,"threads_started":1}
I20260809 12:19:54.409132 7726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.392784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52518 (local address 127.19.183.254:33259)
0809 12:19:54.393956 (+ 1172us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:54.393961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:54.393975 (+ 14us) server_negotiation.cc:407] Connection header received
0809 12:19:54.394028 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:54.394031 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:54.394108 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:54.394231 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:54.395446 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.396424 (+ 978us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.398166 (+ 1742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.398778 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.398900 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:54.402915 (+ 4015us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:54.402922 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:54.402926 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:54.402965 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:54.405971 (+ 3006us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.406818 (+ 847us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.406824 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.406827 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.407017 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.407418 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.407423 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.407427 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.407814 (+ 387us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:54.407998 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:54.408548 (+ 550us) server_negotiation.cc:299] Negotiation successful
0809 12:19:54.408726 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1077,"thread_start_us":985,"threads_started":1}
I20260809 12:19:54.409157 7725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.392722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:33259 (local address 127.0.0.1:52518)
0809 12:19:54.393423 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:54.393462 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 12:19:54.393590 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:19:54.394296 (+ 706us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:19:54.394304 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:19:54.394957 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:19:54.395264 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:54.395294 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.396614 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.396618 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:54.397915 (+ 1297us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:54.397935 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.399150 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.399156 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:54.399344 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:54.400046 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:19:54.400072 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:19:54.402626 (+ 2554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:19:54.406098 (+ 3472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:54.406107 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:54.406113 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:54.406664 (+ 551us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:54.407138 (+ 474us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:54.407142 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:54.407146 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:54.407289 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:54.407961 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:19:54.407969 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:19:54.408187 (+ 218us) client_negotiation.cc:769] Sending connection context
0809 12:19:54.408417 (+ 230us) client_negotiation.cc:240] Negotiation successful
0809 12:19:54.408721 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":461,"thread_start_us":326,"threads_started":1}
I20260809 12:19:54.410578 7720 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:19:54 dist-test-slave-39l8 krb5kdc[7373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
Aug 09 12:19:54 dist-test-slave-39l8 krb5kdc[7373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277994, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.252@KRBTEST.COM for kudu/127.19.183.253@KRBTEST.COM
I20260809 12:19:54.425030 7727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.407874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52522 (local address 127.19.183.254:33259)
0809 12:19:54.408618 (+ 744us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:54.408624 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:54.408647 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:19:54.408696 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:54.408700 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:54.408766 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:54.408878 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:54.410479 (+ 1601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.411583 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.413283 (+ 1700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.413901 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.413987 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:54.419324 (+ 5337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:54.419329 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:54.419332 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:54.419363 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:54.421956 (+ 2593us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.422836 (+ 880us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.422841 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.422844 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.422996 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.423405 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.423411 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.423417 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.423771 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:54.424067 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:54.424338 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 12:19:54.424594 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":348,"threads_started":1}
I20260809 12:19:54.425635 7606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.411365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41054 (local address 127.19.183.253:41169)
0809 12:19:54.411599 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:54.411605 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:54.411624 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:19:54.411686 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:54.411693 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:54.411772 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:54.411844 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:54.412768 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.413616 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.415260 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.415658 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.415749 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:54.419552 (+ 3803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:54.419557 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:54.419560 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:54.419591 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:54.422747 (+ 3156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.423473 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.423477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.423479 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.423608 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.424023 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.424027 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.424031 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.424401 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:54.424618 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:54.425030 (+ 412us) server_negotiation.cc:299] Negotiation successful
0809 12:19:54.425280 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 12:19:54.425714 7725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.411246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:41169 (local address 127.0.0.1:41054)
0809 12:19:54.411391 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:54.411411 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:19:54.411503 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:19:54.411906 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:19:54.411912 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:19:54.412390 (+ 478us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:19:54.412566 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:54.412577 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.413765 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.413769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:54.415090 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:54.415110 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.415826 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.415836 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:54.416074 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:54.416595 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:19:54.416633 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:19:54.419356 (+ 2723us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:19:54.422878 (+ 3522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:54.422885 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:54.422890 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:54.423357 (+ 467us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:54.423725 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:54.423730 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:54.423734 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:54.423907 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:54.424589 (+ 682us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:19:54.424594 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:19:54.424762 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 12:19:54.424919 (+ 157us) client_negotiation.cc:240] Negotiation successful
0809 12:19:54.425263 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 12:19:54.427330 7720 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:19:54 dist-test-slave-39l8 krb5kdc[7373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277994, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.252@KRBTEST.COM for kudu/127.19.183.252@KRBTEST.COM
Aug 09 12:19:54 dist-test-slave-39l8 krb5kdc[7373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277992, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.253@KRBTEST.COM
I20260809 12:19:54.440752 7725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.428155 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:34787 (local address 127.0.0.1:50918)
0809 12:19:54.428313 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:54.428331 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:19:54.428419 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:19:54.428726 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:19:54.428729 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:19:54.429042 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:19:54.429197 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:54.429205 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.430139 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.430147 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:54.431097 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:54.431108 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.431849 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.431855 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:54.432035 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:54.432618 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:19:54.432644 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:19:54.434633 (+ 1989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:19:54.438027 (+ 3394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:54.438030 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:54.438033 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:54.438362 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:54.438884 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:54.438888 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:54.438890 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:54.439016 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:54.439634 (+ 618us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:19:54.439639 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:19:54.439811 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 12:19:54.439988 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 12:19:54.440263 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 12:19:54.440831 7724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.428275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50918 (local address 127.19.183.252:34787)
0809 12:19:54.428452 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:54.428458 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:54.428475 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:19:54.428529 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:54.428536 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:54.428613 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:54.428726 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:54.429336 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.430009 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.431229 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.431680 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.431810 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:54.434853 (+ 3043us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:54.434858 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:54.434861 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:54.434890 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:54.437895 (+ 3005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.438489 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.438494 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.438497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.438671 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.439134 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.439141 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.439147 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.439483 (+ 336us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:54.439588 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:54.440129 (+ 541us) server_negotiation.cc:299] Negotiation successful
0809 12:19:54.440465 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":69,"server-negotiator.queue_time_us":65}
I20260809 12:19:54.442232 7606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.427053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41066 (local address 127.19.183.253:41169)
0809 12:19:54.427208 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:54.427213 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:54.427383 (+ 170us) server_negotiation.cc:407] Connection header received
0809 12:19:54.427605 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:54.427611 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:54.427682 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:54.427781 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:54.429328 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.430042 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.431361 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.431820 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.431958 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:54.436217 (+ 4259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:54.436225 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:54.436230 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:54.436265 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:54.439277 (+ 3012us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.440048 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.440053 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.440056 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.440244 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.440691 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.440696 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.440700 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.441060 (+ 360us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:54.441379 (+ 319us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:54.441703 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 12:19:54.441859 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 12:19:54.446426 7720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7: Bootstrap starting.
I20260809 12:19:54.447433 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:19:54.449788 7720 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7: Neither blocks nor log segments found. Creating new log.
I20260809 12:19:54.450932 7720 log.cc:824] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7: Log is configured to *not* fsync() on all Append() calls
I20260809 12:19:54.451371 7495 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } attempt: 4
I20260809 12:19:54.455273 7720 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7: No bootstrap required, opened a new log
Aug 09 12:19:54 dist-test-slave-39l8 krb5kdc[7373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277993, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for kudu/127.19.183.252@KRBTEST.COM
I20260809 12:19:54.471591 7720 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bd69cd65694cb6a00995695e1c1a68" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33259 } } peers { permanent_uuid: "c7df1a80f5b84eb6976d78f2a387ebc1" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } } peers { permanent_uuid: "0ea541d65a024698ad6dce134e03c7a7" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } }
I20260809 12:19:54.472218 7497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.452037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:34787 (local address 127.0.0.1:50932)
0809 12:19:54.452273 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:54.452294 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:19:54.452408 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:19:54.452801 (+ 393us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:19:54.452810 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:19:54.453351 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:19:54.453618 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:54.453634 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.455472 (+ 1838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.455481 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:54.457297 (+ 1816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:54.457316 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.458615 (+ 1299us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.458624 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:54.458888 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:54.459570 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:19:54.459597 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:19:54.462907 (+ 3310us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:19:54.468938 (+ 6031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:54.468947 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:54.468954 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:54.469513 (+ 559us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:54.470035 (+ 522us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:54.470040 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:54.470044 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:54.470264 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:54.471090 (+ 826us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:19:54.471107 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:19:54.471294 (+ 187us) client_negotiation.cc:769] Sending connection context
0809 12:19:54.471456 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 12:19:54.471757 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106}
I20260809 12:19:54.472241 7720 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:19:54.472394 7720 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ea541d65a024698ad6dce134e03c7a7, State: Initialized, Role: FOLLOWER
I20260809 12:19:54.473474 7720 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bd69cd65694cb6a00995695e1c1a68" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33259 } } peers { permanent_uuid: "c7df1a80f5b84eb6976d78f2a387ebc1" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } } peers { permanent_uuid: "0ea541d65a024698ad6dce134e03c7a7" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } }
I20260809 12:19:54.473805 7724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.452082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50932 (local address 127.19.183.252:34787)
0809 12:19:54.452325 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:54.452332 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:54.452352 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:19:54.452567 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:54.452572 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:54.452662 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:54.452757 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:54.453808 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.455189 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.457525 (+ 2336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.458424 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.459261 (+ 837us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:54.463194 (+ 3933us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:54.463204 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:54.463208 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:54.463247 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:54.468776 (+ 5529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.469692 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.469700 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.469706 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.469878 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.470438 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.470445 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.470449 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.470915 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:54.472962 (+ 2047us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:54.473115 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 12:19:54.473357 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104}
I20260809 12:19:54.480374 7495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68: Bootstrap starting.
I20260809 12:19:54.484972 7495 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68: Neither blocks nor log segments found. Creating new log.
I20260809 12:19:54.486539 7495 log.cc:824] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68: Log is configured to *not* fsync() on all Append() calls
I20260809 12:19:54.490882 7742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bd69cd65694cb6a00995695e1c1a68" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33259 } } peers { permanent_uuid: "c7df1a80f5b84eb6976d78f2a387ebc1" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } } peers { permanent_uuid: "0ea541d65a024698ad6dce134e03c7a7" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } } }
I20260809 12:19:54.491746 7742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:19:54.492013 7742 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:19:54.492304 7742 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bd69cd65694cb6a00995695e1c1a68" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33259 } } peers { permanent_uuid: "c7df1a80f5b84eb6976d78f2a387ebc1" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } } peers { permanent_uuid: "0ea541d65a024698ad6dce134e03c7a7" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } }
I20260809 12:19:54.496474 7495 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68: No bootstrap required, opened a new log
I20260809 12:19:54.500875 7738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.453137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50942 (local address 127.19.183.252:34787)
0809 12:19:54.468702 (+ 15565us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:54.468709 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:54.468733 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:19:54.468785 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:54.468789 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:54.468878 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:54.471550 (+ 2672us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:54.471848 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.473164 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.475235 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.475960 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.479725 (+ 3765us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:54.479983 (+ 258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:54.479989 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:54.479993 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:54.480028 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:54.483630 (+ 3602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.486416 (+ 2786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.486421 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.486442 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.486655 (+ 213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.489032 (+ 2377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.489040 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.489046 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.489530 (+ 484us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:54.497542 (+ 8012us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:54.497684 (+ 142us) server_negotiation.cc:299] Negotiation successful
0809 12:19:54.499502 (+ 1818us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15389,"thread_start_us":15860,"threads_started":1}
I20260809 12:19:54.501499 7742 leader_election.cc:288] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 47bd69cd65694cb6a00995695e1c1a68 (127.19.183.254:33259), c7df1a80f5b84eb6976d78f2a387ebc1 (127.19.183.253:41169)
I20260809 12:19:54.504454 7720 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:19:54.508384 7495 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bd69cd65694cb6a00995695e1c1a68" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33259 } } peers { permanent_uuid: "c7df1a80f5b84eb6976d78f2a387ebc1" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } } peers { permanent_uuid: "0ea541d65a024698ad6dce134e03c7a7" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } }
I20260809 12:19:54.508793 7495 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:19:54.508946 7495 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47bd69cd65694cb6a00995695e1c1a68, State: Initialized, Role: FOLLOWER
I20260809 12:19:54.509678 7495 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bd69cd65694cb6a00995695e1c1a68" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33259 } } peers { permanent_uuid: "c7df1a80f5b84eb6976d78f2a387ebc1" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } } peers { permanent_uuid: "0ea541d65a024698ad6dce134e03c7a7" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } }
I20260809 12:19:54.511195 7751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bd69cd65694cb6a00995695e1c1a68" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33259 } } peers { permanent_uuid: "c7df1a80f5b84eb6976d78f2a387ebc1" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } } peers { permanent_uuid: "0ea541d65a024698ad6dce134e03c7a7" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } } }
I20260809 12:19:54.512243 7751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:19:54.512363 7457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ea541d65a024698ad6dce134e03c7a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47bd69cd65694cb6a00995695e1c1a68" is_pre_election: true
I20260809 12:19:54.512981 7457 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ea541d65a024698ad6dce134e03c7a7 in term 0.
I20260809 12:19:54.512343 7495 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:19:54.513839 7641 leader_election.cc:302] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ea541d65a024698ad6dce134e03c7a7, 47bd69cd65694cb6a00995695e1c1a68; no voters:
I20260809 12:19:54.514401 7742 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:19:54.514582 7742 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:19:54.514739 7742 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:19:54.519204 7742 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bd69cd65694cb6a00995695e1c1a68" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33259 } } peers { permanent_uuid: "c7df1a80f5b84eb6976d78f2a387ebc1" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } } peers { permanent_uuid: "0ea541d65a024698ad6dce134e03c7a7" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } }
I20260809 12:19:54.520905 7457 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ea541d65a024698ad6dce134e03c7a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "47bd69cd65694cb6a00995695e1c1a68"
I20260809 12:19:54.521188 7457 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:19:54.526818 7457 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ea541d65a024698ad6dce134e03c7a7 in term 1.
I20260809 12:19:54.528724 7641 leader_election.cc:302] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ea541d65a024698ad6dce134e03c7a7, 47bd69cd65694cb6a00995695e1c1a68; no voters:
I20260809 12:19:54.529140 7739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:19:54.530426 7739 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [term 1 LEADER]: Becoming Leader. State: Replica: 0ea541d65a024698ad6dce134e03c7a7, State: Running, Role: LEADER
I20260809 12:19:54.530432 7742 leader_election.cc:288] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [CANDIDATE]: Term 1 election: Requested vote from peers 47bd69cd65694cb6a00995695e1c1a68 (127.19.183.254:33259), c7df1a80f5b84eb6976d78f2a387ebc1 (127.19.183.253:41169)
I20260809 12:19:54.531178 7739 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bd69cd65694cb6a00995695e1c1a68" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33259 } } peers { permanent_uuid: "c7df1a80f5b84eb6976d78f2a387ebc1" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } } peers { permanent_uuid: "0ea541d65a024698ad6dce134e03c7a7" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } }
I20260809 12:19:54.532786 7606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.484549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41078 (local address 127.19.183.253:41169)
0809 12:19:54.484787 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:54.484796 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:54.491344 (+ 6548us) server_negotiation.cc:407] Connection header received
0809 12:19:54.491754 (+ 410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:54.491759 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:54.491849 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:54.491939 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:54.494150 (+ 2211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.495302 (+ 1152us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.502756 (+ 7454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.503432 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.503536 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:54.515295 (+ 11759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:54.515305 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:54.515313 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:54.515388 (+ 75us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:54.520166 (+ 4778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.527806 (+ 7640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.527816 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.527822 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.528059 (+ 237us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.528678 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.528687 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.528693 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.529188 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:54.529469 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:54.532049 (+ 2580us) server_negotiation.cc:299] Negotiation successful
0809 12:19:54.532315 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 12:19:54.535997 7727 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.452969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52532 (local address 127.19.183.254:33259)
0809 12:19:54.455439 (+ 2470us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:54.455446 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:54.484317 (+ 28871us) server_negotiation.cc:407] Connection header received
0809 12:19:54.484658 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:54.484666 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:54.484763 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:54.484850 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:54.497995 (+ 13145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.499345 (+ 1350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.506143 (+ 6798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.506955 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.507058 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:54.521455 (+ 14397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:54.521466 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:54.521473 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:54.521530 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:54.526447 (+ 4917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.530068 (+ 3621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.530076 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.530080 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.530250 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.530778 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.530783 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.530786 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.531133 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:54.531372 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:54.535290 (+ 3918us) server_negotiation.cc:299] Negotiation successful
0809 12:19:54.535570 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2334}
I20260809 12:19:54.536144 7742 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ea541d65a024698ad6dce134e03c7a7. Latest consensus state: current_term: 1 leader_uuid: "0ea541d65a024698ad6dce134e03c7a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bd69cd65694cb6a00995695e1c1a68" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33259 } } peers { permanent_uuid: "c7df1a80f5b84eb6976d78f2a387ebc1" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } } peers { permanent_uuid: "0ea541d65a024698ad6dce134e03c7a7" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } } }
I20260809 12:19:54.536671 7742 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [sys.catalog]: This master's current role is: LEADER
I20260809 12:19:54.541970 7760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:19:54.555022 7760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260809 12:19:54.551441 7427 tablet.cc:2369] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:19:54.560070 7600 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } attempt: 5
I20260809 12:19:54.569897 7457 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68 [term 1 FOLLOWER]: Refusing update from remote peer 0ea541d65a024698ad6dce134e03c7a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:19:54.571532 7757 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "47bd69cd65694cb6a00995695e1c1a68" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Aug 09 12:19:54 dist-test-slave-39l8 krb5kdc[7373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277993, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.253@KRBTEST.COM for kudu/127.19.183.252@KRBTEST.COM
I20260809 12:19:54.563180 7610 master_runner.cc:186] Error getting master registration for 127.19.183.254:33259: OK, instance_id { permanent_uuid: "47bd69cd65694cb6a00995695e1c1a68" instance_seqno: 1786277993312090 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 12:19:54.576359 7610 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 12:19:54.581080 7605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.560747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:34787 (local address 127.0.0.1:50958)
0809 12:19:54.561077 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:54.561103 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 12:19:54.561259 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:19:54.564039 (+ 2780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:19:54.564045 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:19:54.564623 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:19:54.564833 (+ 210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:54.564845 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.566386 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.566394 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:54.567876 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:19:54.567894 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.569049 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.569055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:19:54.569277 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:54.569947 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:19:54.569968 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:19:54.572880 (+ 2912us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:19:54.577434 (+ 4554us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:54.577441 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:54.577445 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:54.577935 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:54.578679 (+ 744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:19:54.578684 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:19:54.578688 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:19:54.578899 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:19:54.580116 (+ 1217us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:19:54.580133 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:19:54.580300 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 12:19:54.580432 (+ 132us) client_negotiation.cc:240] Negotiation successful
0809 12:19:54.580711 (+ 279us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155}
I20260809 12:19:54.582062 7738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:54.563354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50958 (local address 127.19.183.252:34787)
0809 12:19:54.563543 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:54.563549 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:54.563567 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:19:54.563625 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:54.563636 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:54.563733 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:54.563841 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:54.565015 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.566183 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.568043 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:54.568730 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:54.568860 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:54.573113 (+ 4253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:54.573130 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:54.573135 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:54.573181 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:54.577273 (+ 4092us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.578130 (+ 857us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.578136 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.578140 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.578471 (+ 331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:54.579035 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:54.579040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:54.579043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:54.579910 (+ 867us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:54.580059 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:54.581515 (+ 1456us) server_negotiation.cc:299] Negotiation successful
0809 12:19:54.581734 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:19:54.566356 7396 master_runner.cc:186] Error getting master registration for 127.19.183.253:41169: OK, instance_id { permanent_uuid: "c7df1a80f5b84eb6976d78f2a387ebc1" instance_seqno: 1786277993828591 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 12:19:54.583863 7396 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 12:19:54.588591 7600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7df1a80f5b84eb6976d78f2a387ebc1: Bootstrap starting.
W20260809 12:19:54.590039 7784 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 12:19:54.590283 7784 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 12:19:54.593091 7600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c7df1a80f5b84eb6976d78f2a387ebc1: Neither blocks nor log segments found. Creating new log.
I20260809 12:19:54.594440 7600 log.cc:824] T 00000000000000000000000000000000 P c7df1a80f5b84eb6976d78f2a387ebc1: Log is configured to *not* fsync() on all Append() calls
I20260809 12:19:54.602070 7751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ea541d65a024698ad6dce134e03c7a7. Latest consensus state: current_term: 1 leader_uuid: "0ea541d65a024698ad6dce134e03c7a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bd69cd65694cb6a00995695e1c1a68" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33259 } } peers { permanent_uuid: "c7df1a80f5b84eb6976d78f2a387ebc1" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } } peers { permanent_uuid: "0ea541d65a024698ad6dce134e03c7a7" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } } }
I20260809 12:19:54.605688 7751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:19:54.607100 7600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c7df1a80f5b84eb6976d78f2a387ebc1: No bootstrap required, opened a new log
W20260809 12:19:54.608675 7648 tablet.cc:2369] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:19:54.610064 7783 mvcc.cc:205] Tried to move back new op lower bound from 7316594665739894784 to 7316594665605488640. Current Snapshot: MvccSnapshot[applied={T|T < 7316594665739894784}]
I20260809 12:19:54.611375 7757 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0ea541d65a024698ad6dce134e03c7a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bd69cd65694cb6a00995695e1c1a68" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33259 } } peers { permanent_uuid: "c7df1a80f5b84eb6976d78f2a387ebc1" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } } peers { permanent_uuid: "0ea541d65a024698ad6dce134e03c7a7" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } } }
I20260809 12:19:54.611780 7757 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [sys.catalog]: This master's current role is: LEADER
I20260809 12:19:54.621167 7600 raft_consensus.cc:348] T 00000000000000000000000000000000 P c7df1a80f5b84eb6976d78f2a387ebc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bd69cd65694cb6a00995695e1c1a68" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33259 } } peers { permanent_uuid: "c7df1a80f5b84eb6976d78f2a387ebc1" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } } peers { permanent_uuid: "0ea541d65a024698ad6dce134e03c7a7" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } }
I20260809 12:19:54.621697 7600 raft_consensus.cc:374] T 00000000000000000000000000000000 P c7df1a80f5b84eb6976d78f2a387ebc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:19:54.621874 7600 raft_consensus.cc:729] T 00000000000000000000000000000000 P c7df1a80f5b84eb6976d78f2a387ebc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c7df1a80f5b84eb6976d78f2a387ebc1, State: Initialized, Role: FOLLOWER
I20260809 12:19:54.622465 7751 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0ea541d65a024698ad6dce134e03c7a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bd69cd65694cb6a00995695e1c1a68" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33259 } } peers { permanent_uuid: "c7df1a80f5b84eb6976d78f2a387ebc1" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } } peers { permanent_uuid: "0ea541d65a024698ad6dce134e03c7a7" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } } }
I20260809 12:19:54.622777 7751 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:19:54.622823 7600 consensus_queue.cc:260] T 00000000000000000000000000000000 P c7df1a80f5b84eb6976d78f2a387ebc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bd69cd65694cb6a00995695e1c1a68" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33259 } } peers { permanent_uuid: "c7df1a80f5b84eb6976d78f2a387ebc1" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } } peers { permanent_uuid: "0ea541d65a024698ad6dce134e03c7a7" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } }
I20260809 12:19:54.626165 7760 catalog_manager.cc:1347] Generated new cluster ID: 035f4bbd697449c89a733055beaa3a4c
I20260809 12:19:54.626293 7760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:19:54.639302 7795 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7df1a80f5b84eb6976d78f2a387ebc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bd69cd65694cb6a00995695e1c1a68" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33259 } } peers { permanent_uuid: "c7df1a80f5b84eb6976d78f2a387ebc1" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } } peers { permanent_uuid: "0ea541d65a024698ad6dce134e03c7a7" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } } }
I20260809 12:19:54.639925 7795 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7df1a80f5b84eb6976d78f2a387ebc1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:19:54.641079 7564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ea541d65a024698ad6dce134e03c7a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7df1a80f5b84eb6976d78f2a387ebc1"
I20260809 12:19:54.641618 7563 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c7df1a80f5b84eb6976d78f2a387ebc1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:19:54.641057 7600 sys_catalog.cc:564] T 00000000000000000000000000000000 P c7df1a80f5b84eb6976d78f2a387ebc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:19:54.645962 7563 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c7df1a80f5b84eb6976d78f2a387ebc1 [term 1 FOLLOWER]: Refusing update from remote peer 0ea541d65a024698ad6dce134e03c7a7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:19:54.641134 7565 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0ea541d65a024698ad6dce134e03c7a7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c7df1a80f5b84eb6976d78f2a387ebc1" is_pre_election: true
I20260809 12:19:54.646859 7565 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c7df1a80f5b84eb6976d78f2a387ebc1 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ea541d65a024698ad6dce134e03c7a7 in term 1.
I20260809 12:19:54.646911 7739 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c7df1a80f5b84eb6976d78f2a387ebc1" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:19:54.658924 7760 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:19:54.660923 7760 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:19:54.682868 7760 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7: Generated new TSK 0
I20260809 12:19:54.684106 7760 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:19:54.692418 7795 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7df1a80f5b84eb6976d78f2a387ebc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ea541d65a024698ad6dce134e03c7a7. Latest consensus state: current_term: 1 leader_uuid: "0ea541d65a024698ad6dce134e03c7a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bd69cd65694cb6a00995695e1c1a68" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33259 } } peers { permanent_uuid: "c7df1a80f5b84eb6976d78f2a387ebc1" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } } peers { permanent_uuid: "0ea541d65a024698ad6dce134e03c7a7" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } } }
I20260809 12:19:54.692763 7795 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7df1a80f5b84eb6976d78f2a387ebc1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:19:54.693259 7809 sys_catalog.cc:455] T 00000000000000000000000000000000 P c7df1a80f5b84eb6976d78f2a387ebc1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0ea541d65a024698ad6dce134e03c7a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bd69cd65694cb6a00995695e1c1a68" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33259 } } peers { permanent_uuid: "c7df1a80f5b84eb6976d78f2a387ebc1" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } } peers { permanent_uuid: "0ea541d65a024698ad6dce134e03c7a7" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } } }
I20260809 12:19:54.693670 7809 sys_catalog.cc:458] T 00000000000000000000000000000000 P c7df1a80f5b84eb6976d78f2a387ebc1 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:19:54.695021 7739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0ea541d65a024698ad6dce134e03c7a7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "47bd69cd65694cb6a00995695e1c1a68" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33259 } } peers { permanent_uuid: "c7df1a80f5b84eb6976d78f2a387ebc1" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 41169 } } peers { permanent_uuid: "0ea541d65a024698ad6dce134e03c7a7" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 34787 } } }
I20260809 12:19:54.695328 7739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 [sys.catalog]: This master's current role is: LEADER
I20260809 12:19:54.779305 7826 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 12:19:55.592298 7784 catalog_manager.cc:1259] Loaded cluster ID: 035f4bbd697449c89a733055beaa3a4c
I20260809 12:19:55.592406 7784 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68: loading cluster ID for follower catalog manager: success
I20260809 12:19:55.595000 7784 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68: acquiring CA information for follower catalog manager: success
I20260809 12:19:55.596702 7784 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 47bd69cd65694cb6a00995695e1c1a68: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 12:19:55.784338 7826 catalog_manager.cc:1259] Loaded cluster ID: 035f4bbd697449c89a733055beaa3a4c
I20260809 12:19:55.784483 7826 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c7df1a80f5b84eb6976d78f2a387ebc1: loading cluster ID for follower catalog manager: success
I20260809 12:19:55.787161 7826 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c7df1a80f5b84eb6976d78f2a387ebc1: acquiring CA information for follower catalog manager: success
I20260809 12:19:55.788975 7826 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c7df1a80f5b84eb6976d78f2a387ebc1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 12:19:55.838461 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 7396
I20260809 12:19:55.839707 7483 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:19:55.951387 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 7396
I20260809 12:19:55.993955 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 7503
I20260809 12:19:55.995195 7590 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:19:56.069764 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:56.068947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:33259 (local address 127.0.0.1:52540)
0809 12:19:56.069462 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:56.069603 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:33259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":396,"thread_start_us":305,"threads_started":1}
W20260809 12:19:56.070053 7641 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.183.254:33259: connect: Connection refused (error 111)
I20260809 12:19:56.071569 7831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:56.071223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:33259 (local address 127.0.0.1:52552)
0809 12:19:56.071379 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 12:19:56.071459 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:33259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 12:19:56.071780 7641 consensus_peers.cc:487] T 00000000000000000000000000000000 P 0ea541d65a024698ad6dce134e03c7a7 -> Peer 47bd69cd65694cb6a00995695e1c1a68 (127.19.183.254:33259): Couldn't send request to peer 47bd69cd65694cb6a00995695e1c1a68. Status: Network error: Client connection negotiation failed: client connection to 127.19.183.254:33259: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 12:19:56.104545 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 7503
I20260809 12:19:56.131999 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 7610
I20260809 12:19:56.133087 7711 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:19:56.241143 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 7610
2026-08-09T12:19:56Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (4984 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:19:56 dist-test-slave-39l8 krb5kdc[7845](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:19:56 dist-test-slave-39l8 krb5kdc[7845](info): set up 2 sockets
Aug 09 12:19:56 dist-test-slave-39l8 krb5kdc[7845](info): commencing operation
krb5kdc: starting...
W20260809 12:19:57.523356 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.231s user 0.002s sys 0.032s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:19:57 dist-test-slave-39l8 krb5kdc[7845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:19:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:19:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:19:57.789991 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:38587
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:38581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:38587
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:19:58.129536 7861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:19:58.129810 7861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:19:58.129899 7861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:19:58.136919 7861 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:19:58.137014 7861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:19:58.137061 7861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:19:58.137118 7861 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:19:58.137162 7861 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:19:58.146147 7861 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:38581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:38587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:38587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:19:58.147262 7861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:19:58.148452 7861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:19:58.159866 7871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:19:58.160017 7867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:19:58.159968 7868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:19:58.161463 7861 server_base.cc:1034] running on GCE node
I20260809 12:19:58.162182 7861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:19:58.163527 7861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:19:58.164716 7861 hybrid_clock.cc:648] HybridClock initialized: now 1786277998164666 us; error 47 us; skew 500 ppm
I20260809 12:19:58.165062 7861 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:19:58 dist-test-slave-39l8 krb5kdc[7845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277998, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:19:58.168226 7861 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:19:58.169750 7861 webserver.cc:460] Webserver started at http://127.19.183.254:36005/ using document root <none> and password file <none>
I20260809 12:19:58.170434 7861 fs_manager.cc:359] Metadata directory not provided
I20260809 12:19:58.170583 7861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:19:58.170959 7861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:19:58.174453 7861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "f9f1f78e29dd416ab531ced9dff9a49d"
format_stamp: "Formatted at 2026-08-09 12:19:58 on dist-test-slave-39l8"
server_key: "0bb7c67660468118f1eedd8353d918ca"
server_key_iv: "ddc7c36c47c6aa8db62ed1251b0850cc"
server_key_version: "encryptionkey@0"
I20260809 12:19:58.175556 7861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "f9f1f78e29dd416ab531ced9dff9a49d"
format_stamp: "Formatted at 2026-08-09 12:19:58 on dist-test-slave-39l8"
server_key: "0bb7c67660468118f1eedd8353d918ca"
server_key_iv: "ddc7c36c47c6aa8db62ed1251b0850cc"
server_key_version: "encryptionkey@0"
I20260809 12:19:58.183068 7861 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260809 12:19:58.191082 7886 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:19:58.192564 7861 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.008s sys 0.000s
I20260809 12:19:58.192720 7861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "f9f1f78e29dd416ab531ced9dff9a49d"
format_stamp: "Formatted at 2026-08-09 12:19:58 on dist-test-slave-39l8"
server_key: "0bb7c67660468118f1eedd8353d918ca"
server_key_iv: "ddc7c36c47c6aa8db62ed1251b0850cc"
server_key_version: "encryptionkey@0"
I20260809 12:19:58.192896 7861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:19:58.215315 7861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:19:58.218987 7861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:19:58.219215 7861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:19:58.237854 7861 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:38587
I20260809 12:19:58.237885 7958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:38587 every 8 connection(s)
I20260809 12:19:58.239240 7861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:19:58.241430 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 7861
I20260809 12:19:58.241669 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:19:58.242360 20191 external_mini_cluster.cc:1428] Setting key 219dec5c4a6cab32dbc4f7a979f332e0
I20260809 12:19:58.243053 7959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:19:58.255448 7959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9f1f78e29dd416ab531ced9dff9a49d: Bootstrap starting.
Aug 09 12:19:58 dist-test-slave-39l8 krb5kdc[7845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277997, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:19:58.258874 7959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9f1f78e29dd416ab531ced9dff9a49d: Neither blocks nor log segments found. Creating new log.
I20260809 12:19:58.259985 7959 log.cc:824] T 00000000000000000000000000000000 P f9f1f78e29dd416ab531ced9dff9a49d: Log is configured to *not* fsync() on all Append() calls
I20260809 12:19:58.262914 7959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9f1f78e29dd416ab531ced9dff9a49d: No bootstrap required, opened a new log
I20260809 12:19:58.264281 7963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:19:58.244259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55122 (local address 127.19.183.254:38587)
0809 12:19:58.245064 (+ 805us) server_negotiation.cc:206] Beginning negotiation
0809 12:19:58.245075 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:19:58.245126 (+ 51us) server_negotiation.cc:407] Connection header received
0809 12:19:58.246752 (+ 1626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:19:58.246779 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:19:58.247176 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:19:58.247562 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:19:58.249103 (+ 1541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:58.250416 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:58.251790 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:19:58.252517 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:19:58.252739 (+ 222us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:19:58.257140 (+ 4401us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:19:58.257148 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:19:58.257161 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:19:58.257198 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:19:58.260365 (+ 3167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:58.261097 (+ 732us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:58.261104 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:58.261109 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:58.261291 (+ 182us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:19:58.261686 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:19:58.261690 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:19:58.261695 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:19:58.262194 (+ 499us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:19:58.262492 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0809 12:19:58.262971 (+ 479us) server_negotiation.cc:299] Negotiation successful
0809 12:19:58.263387 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":604,"thread_start_us":376,"threads_started":1}
I20260809 12:19:58.268138 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:19:58.271781 7959 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9f1f78e29dd416ab531ced9dff9a49d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f1f78e29dd416ab531ced9dff9a49d" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38587 } }
I20260809 12:19:58.272198 7959 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9f1f78e29dd416ab531ced9dff9a49d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:19:58.272317 7959 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9f1f78e29dd416ab531ced9dff9a49d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9f1f78e29dd416ab531ced9dff9a49d, State: Initialized, Role: FOLLOWER
I20260809 12:19:58.272917 7959 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9f1f78e29dd416ab531ced9dff9a49d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f1f78e29dd416ab531ced9dff9a49d" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38587 } }
I20260809 12:19:58.273181 7959 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9f1f78e29dd416ab531ced9dff9a49d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:19:58.273316 7959 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9f1f78e29dd416ab531ced9dff9a49d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:19:58.273499 7959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9f1f78e29dd416ab531ced9dff9a49d [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:19:58.275693 7959 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9f1f78e29dd416ab531ced9dff9a49d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f1f78e29dd416ab531ced9dff9a49d" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38587 } }
I20260809 12:19:58.276201 7959 leader_election.cc:302] T 00000000000000000000000000000000 P f9f1f78e29dd416ab531ced9dff9a49d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9f1f78e29dd416ab531ced9dff9a49d; no voters:
I20260809 12:19:58.276786 7959 leader_election.cc:288] T 00000000000000000000000000000000 P f9f1f78e29dd416ab531ced9dff9a49d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:19:58.276917 7966 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9f1f78e29dd416ab531ced9dff9a49d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:19:58.277624 7966 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9f1f78e29dd416ab531ced9dff9a49d [term 1 LEADER]: Becoming Leader. State: Replica: f9f1f78e29dd416ab531ced9dff9a49d, State: Running, Role: LEADER
I20260809 12:19:58.278147 7966 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9f1f78e29dd416ab531ced9dff9a49d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f1f78e29dd416ab531ced9dff9a49d" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38587 } }
I20260809 12:19:58.278868 7959 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9f1f78e29dd416ab531ced9dff9a49d [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:19:58.280903 7967 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9f1f78e29dd416ab531ced9dff9a49d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9f1f78e29dd416ab531ced9dff9a49d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f1f78e29dd416ab531ced9dff9a49d" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38587 } } }
I20260809 12:19:58.281235 7967 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9f1f78e29dd416ab531ced9dff9a49d [sys.catalog]: This master's current role is: LEADER
I20260809 12:19:58.283114 7973 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:19:58.282967 7968 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9f1f78e29dd416ab531ced9dff9a49d [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9f1f78e29dd416ab531ced9dff9a49d. Latest consensus state: current_term: 1 leader_uuid: "f9f1f78e29dd416ab531ced9dff9a49d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f1f78e29dd416ab531ced9dff9a49d" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38587 } } }
I20260809 12:19:58.283265 7968 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9f1f78e29dd416ab531ced9dff9a49d [sys.catalog]: This master's current role is: LEADER
I20260809 12:19:58.288093 7973 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:19:58.297166 7973 catalog_manager.cc:1347] Generated new cluster ID: 886c52039eb543e88ae694d326fce79b
I20260809 12:19:58.297271 7973 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:19:58.319278 7973 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:19:58.320503 7973 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:19:58.332581 7973 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9f1f78e29dd416ab531ced9dff9a49d: Generated new TSK 0
I20260809 12:19:58.333412 7973 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:19:58.382248 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 7861
I20260809 12:19:58.384533 7951 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:19:58.481696 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 7861
2026-08-09T12:19:58Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (2235 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:19:58 dist-test-slave-39l8 krb5kdc[7994](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:19:58 dist-test-slave-39l8 krb5kdc[7994](info): set up 2 sockets
Aug 09 12:19:58 dist-test-slave-39l8 krb5kdc[7994](info): commencing operation
krb5kdc: starting...
W20260809 12:19:59.769398 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:19:59 dist-test-slave-39l8 krb5kdc[7994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:19:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:19:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:20:00.035374 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:45033
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:46539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:45033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:20:00.374917 8019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:20:00.375190 8019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:20:00.375381 8019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:20:00.382423 8019 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:20:00.382545 8019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:20:00.382634 8019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:20:00.382755 8019 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:20:00.382846 8019 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:20:00.392098 8019 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:46539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:45033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:45033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:20:00.393291 8019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:20:00.394518 8019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:20:00.405468 8029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:00.405529 8025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:00.405556 8027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:20:00.406365 8019 server_base.cc:1034] running on GCE node
I20260809 12:20:00.407153 8019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:20:00.408435 8019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:20:00.409633 8019 hybrid_clock.cc:648] HybridClock initialized: now 1786278000409593 us; error 34 us; skew 500 ppm
I20260809 12:20:00.409987 8019 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:20:00 dist-test-slave-39l8 krb5kdc[7994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278000, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:20:00.413137 8019 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:20:00.414650 8019 webserver.cc:460] Webserver started at http://127.19.183.254:35351/ using document root <none> and password file <none>
I20260809 12:20:00.415373 8019 fs_manager.cc:359] Metadata directory not provided
I20260809 12:20:00.415508 8019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:20:00.415822 8019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:20:00.418632 8019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "c8a0592aa088438ea86a0bbb545d93c0"
format_stamp: "Formatted at 2026-08-09 12:20:00 on dist-test-slave-39l8"
server_key: "bdf58f8d2789001a231b04dfb4fc1d30"
server_key_iv: "04357a4fa0b65ba1a45d16c7cbaf5c2a"
server_key_version: "encryptionkey@0"
I20260809 12:20:00.419435 8019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "c8a0592aa088438ea86a0bbb545d93c0"
format_stamp: "Formatted at 2026-08-09 12:20:00 on dist-test-slave-39l8"
server_key: "bdf58f8d2789001a231b04dfb4fc1d30"
server_key_iv: "04357a4fa0b65ba1a45d16c7cbaf5c2a"
server_key_version: "encryptionkey@0"
I20260809 12:20:00.425261 8019 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 12:20:00.428562 8041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:00.429790 8019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:20:00.429960 8019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "c8a0592aa088438ea86a0bbb545d93c0"
format_stamp: "Formatted at 2026-08-09 12:20:00 on dist-test-slave-39l8"
server_key: "bdf58f8d2789001a231b04dfb4fc1d30"
server_key_iv: "04357a4fa0b65ba1a45d16c7cbaf5c2a"
server_key_version: "encryptionkey@0"
I20260809 12:20:00.430128 8019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:20:00.454468 8019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:20:00.458920 8019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:20:00.459209 8019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:20:00.477798 8019 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:45033
I20260809 12:20:00.477837 8112 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:45033 every 8 connection(s)
I20260809 12:20:00.479172 8019 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:20:00.482702 8113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:20:00.486016 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 8019
I20260809 12:20:00.486243 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:20:00.487159 20191 external_mini_cluster.cc:1428] Setting key 97dfa5a70da32a3009312ef59ed6371a
I20260809 12:20:00.495651 8113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0: Bootstrap starting.
I20260809 12:20:00.499334 8113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0: Neither blocks nor log segments found. Creating new log.
I20260809 12:20:00.500638 8113 log.cc:824] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0: Log is configured to *not* fsync() on all Append() calls
Aug 09 12:20:00 dist-test-slave-39l8 krb5kdc[7994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:20:00.503698 8113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0: No bootstrap required, opened a new log
I20260809 12:20:00.510378 8117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:00.489184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48662 (local address 127.19.183.254:45033)
0809 12:20:00.489900 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:00.489911 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:00.489950 (+ 39us) server_negotiation.cc:407] Connection header received
0809 12:20:00.491612 (+ 1662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:00.491633 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:00.491978 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:00.492317 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:00.494023 (+ 1706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:00.495368 (+ 1345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:00.497186 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:00.497811 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:00.497935 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:00.503021 (+ 5086us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:00.503028 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:00.503042 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:00.503080 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:00.506301 (+ 3221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:00.507218 (+ 917us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:00.507224 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:00.507230 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:00.507418 (+ 188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:00.507846 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:00.507849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:00.507851 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:00.508343 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:00.508580 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:00.509008 (+ 428us) server_negotiation.cc:299] Negotiation successful
0809 12:20:00.509353 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":525,"thread_start_us":324,"threads_started":1}
I20260809 12:20:00.512809 8113 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0592aa088438ea86a0bbb545d93c0" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45033 } }
I20260809 12:20:00.513257 8113 raft_consensus.cc:374] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:20:00.513366 8113 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8a0592aa088438ea86a0bbb545d93c0, State: Initialized, Role: FOLLOWER
I20260809 12:20:00.513900 8113 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0592aa088438ea86a0bbb545d93c0" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45033 } }
I20260809 12:20:00.514118 8113 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:20:00.514273 8113 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:20:00.514430 8113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:20:00.516850 8113 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0592aa088438ea86a0bbb545d93c0" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45033 } }
I20260809 12:20:00.517320 8113 leader_election.cc:302] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8a0592aa088438ea86a0bbb545d93c0; no voters:
I20260809 12:20:00.517910 8113 leader_election.cc:288] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:20:00.518061 8120 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:20:00.518858 8120 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 1 LEADER]: Becoming Leader. State: Replica: c8a0592aa088438ea86a0bbb545d93c0, State: Running, Role: LEADER
I20260809 12:20:00.519513 8120 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0592aa088438ea86a0bbb545d93c0" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45033 } }
I20260809 12:20:00.520141 8113 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:20:00.522300 8121 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c8a0592aa088438ea86a0bbb545d93c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0592aa088438ea86a0bbb545d93c0" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45033 } } }
I20260809 12:20:00.522526 8124 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8a0592aa088438ea86a0bbb545d93c0. Latest consensus state: current_term: 1 leader_uuid: "c8a0592aa088438ea86a0bbb545d93c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0592aa088438ea86a0bbb545d93c0" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45033 } } }
I20260809 12:20:00.522831 8121 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [sys.catalog]: This master's current role is: LEADER
I20260809 12:20:00.522837 8124 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [sys.catalog]: This master's current role is: LEADER
I20260809 12:20:00.524560 8131 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:20:00.529706 8131 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:20:00.538208 8131 catalog_manager.cc:1347] Generated new cluster ID: 54a46fa9988e4b1896acc48c1d2a09d1
I20260809 12:20:00.538317 8131 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:20:00.569823 8131 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:20:00.571475 8131 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:20:00.578644 8131 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0: Generated new TSK 0
I20260809 12:20:00.579435 8131 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
I20260809 12:20:00.650166 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:45033
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:46539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:20:00.992486 8151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:20:00.992817 8151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:20:00.993006 8151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:20:01.000049 8151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:20:01.000208 8151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:20:01.000394 8151 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:20:01.010730 8151 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:46539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:45033
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:20:01.012410 8151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:20:01.014254 8151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:20:01.028630 8163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:01.028746 8158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:01.028724 8159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:20:01.029528 8151 server_base.cc:1034] running on GCE node
I20260809 12:20:01.030117 8151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:20:01.031133 8151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:20:01.032322 8151 hybrid_clock.cc:648] HybridClock initialized: now 1786278001032264 us; error 52 us; skew 500 ppm
I20260809 12:20:01.032645 8151 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:20:01 dist-test-slave-39l8 krb5kdc[7994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278001, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:20:01.035960 8151 init.cc:377] Logged in from keytab as kudu/127.19.183.193@KRBTEST.COM (short username kudu)
I20260809 12:20:01.037457 8151 webserver.cc:460] Webserver started at http://127.19.183.193:42227/ using document root <none> and password file <none>
I20260809 12:20:01.038146 8151 fs_manager.cc:359] Metadata directory not provided
I20260809 12:20:01.038278 8151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:20:01.038589 8151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:20:01.041687 8151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "4f3e4b638b13408a95ef66cdb91b2865"
format_stamp: "Formatted at 2026-08-09 12:20:01 on dist-test-slave-39l8"
server_key: "60ccc26f4fe3ebb0d68f6e558d5f911c"
server_key_iv: "277f1781e4341768f7e8f7490ac58b51"
server_key_version: "encryptionkey@0"
I20260809 12:20:01.042524 8151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "4f3e4b638b13408a95ef66cdb91b2865"
format_stamp: "Formatted at 2026-08-09 12:20:01 on dist-test-slave-39l8"
server_key: "60ccc26f4fe3ebb0d68f6e558d5f911c"
server_key_iv: "277f1781e4341768f7e8f7490ac58b51"
server_key_version: "encryptionkey@0"
I20260809 12:20:01.047876 8151 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 12:20:01.051395 8176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:01.052695 8151 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:20:01.052858 8151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "4f3e4b638b13408a95ef66cdb91b2865"
format_stamp: "Formatted at 2026-08-09 12:20:01 on dist-test-slave-39l8"
server_key: "60ccc26f4fe3ebb0d68f6e558d5f911c"
server_key_iv: "277f1781e4341768f7e8f7490ac58b51"
server_key_version: "encryptionkey@0"
I20260809 12:20:01.053056 8151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:20:01.073184 8151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:20:01.077113 8151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:20:01.077396 8151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:20:01.078397 8151 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:20:01.079852 8151 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:20:01.079939 8151 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:01.080029 8151 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:20:01.080081 8151 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:01.114542 8151 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:43569
I20260809 12:20:01.114590 8342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:43569 every 8 connection(s)
I20260809 12:20:01.115965 8151 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:20:01.119701 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 8151
I20260809 12:20:01.119935 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:20:01.120851 20191 external_mini_cluster.cc:1428] Setting key 4ae6e84565c9c19afca5447fa775bb36
Aug 09 12:20:01 dist-test-slave-39l8 krb5kdc[7994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278001, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:20:01.139590 8345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:01.118584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:53429 (local address 127.19.183.254:45033)
0809 12:20:01.119077 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:01.119082 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:01.120087 (+ 1005us) server_negotiation.cc:407] Connection header received
0809 12:20:01.122533 (+ 2446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:01.122537 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:01.122611 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:01.122746 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:01.124310 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:01.125502 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:01.127155 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:01.127909 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:01.128056 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:01.132568 (+ 4512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:01.132576 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:01.132584 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:01.132627 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:01.136073 (+ 3446us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:01.137076 (+ 1003us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:01.137080 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:01.137082 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:01.137194 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:01.137698 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:01.137701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:01.137703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:01.137956 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:01.138087 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:01.138842 (+ 755us) server_negotiation.cc:299] Negotiation successful
0809 12:20:01.139093 (+ 251us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":269,"threads_started":1}
I20260809 12:20:01.140282 8346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:01.119012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.193:53429)
0809 12:20:01.119891 (+ 879us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:01.119956 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:01.122236 (+ 2280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:01.123021 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:01.123032 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:01.123581 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:20:01.124130 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:01.124147 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:01.125661 (+ 1514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:01.125665 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:01.126998 (+ 1333us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:01.127010 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:01.128140 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:01.128148 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:01.128437 (+ 289us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:01.129231 (+ 794us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:20:01.129262 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:20:01.132326 (+ 3064us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:20:01.136229 (+ 3903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:01.136241 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:01.136260 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:01.136946 (+ 686us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:01.137299 (+ 353us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:01.137303 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:01.137308 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:01.137564 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:01.138067 (+ 503us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:20:01.138073 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:20:01.138427 (+ 354us) client_negotiation.cc:769] Sending connection context
0809 12:20:01.138653 (+ 226us) client_negotiation.cc:240] Negotiation successful
0809 12:20:01.139101 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":310,"threads_started":1}
I20260809 12:20:01.141920 8343 heartbeater.cc:347] Connected to a master server at 127.19.183.254:45033
I20260809 12:20:01.142220 8343 heartbeater.cc:464] Registering TS with master...
I20260809 12:20:01.142992 8343 heartbeater.cc:511] Master 127.19.183.254:45033 requested a full tablet report, sending...
I20260809 12:20:01.145359 8065 ts_manager.cc:194] Registered new tserver with Master: 4f3e4b638b13408a95ef66cdb91b2865 (127.19.183.193:43569)
I20260809 12:20:01.147527 8065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:53429
WARNING: no policy specified for kudu/127.19.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
I20260809 12:20:01.187893 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:45033
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:46539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:20:01.522455 8355 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:20:01.522787 8355 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:20:01.522905 8355 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:20:01.530118 8355 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:20:01.530222 8355 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:20:01.530341 8355 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:20:01.539880 8355 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:46539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:45033
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:20:01.541060 8355 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:20:01.542274 8355 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:20:01.555517 8369 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:01.555603 8366 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:01.555589 8364 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:20:01.556895 8355 server_base.cc:1034] running on GCE node
I20260809 12:20:01.557622 8355 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:20:01.558502 8355 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:20:01.559711 8355 hybrid_clock.cc:648] HybridClock initialized: now 1786278001559650 us; error 50 us; skew 500 ppm
I20260809 12:20:01.559980 8355 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:20:01 dist-test-slave-39l8 krb5kdc[7994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278001, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:20:01.563206 8355 init.cc:377] Logged in from keytab as kudu/127.19.183.194@KRBTEST.COM (short username kudu)
I20260809 12:20:01.564492 8355 webserver.cc:460] Webserver started at http://127.19.183.194:36611/ using document root <none> and password file <none>
I20260809 12:20:01.565193 8355 fs_manager.cc:359] Metadata directory not provided
I20260809 12:20:01.565330 8355 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:20:01.565636 8355 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:20:01.568645 8355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "819e376295694741baea9fe748936978"
format_stamp: "Formatted at 2026-08-09 12:20:01 on dist-test-slave-39l8"
server_key: "411bf0dd8e189818b91170f619d9139c"
server_key_iv: "cea7bb78442f32bdd02a4805ca12cb0b"
server_key_version: "encryptionkey@0"
I20260809 12:20:01.569434 8355 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "819e376295694741baea9fe748936978"
format_stamp: "Formatted at 2026-08-09 12:20:01 on dist-test-slave-39l8"
server_key: "411bf0dd8e189818b91170f619d9139c"
server_key_iv: "cea7bb78442f32bdd02a4805ca12cb0b"
server_key_version: "encryptionkey@0"
I20260809 12:20:01.575004 8355 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 12:20:01.578428 8376 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:01.579816 8355 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:20:01.579988 8355 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "819e376295694741baea9fe748936978"
format_stamp: "Formatted at 2026-08-09 12:20:01 on dist-test-slave-39l8"
server_key: "411bf0dd8e189818b91170f619d9139c"
server_key_iv: "cea7bb78442f32bdd02a4805ca12cb0b"
server_key_version: "encryptionkey@0"
I20260809 12:20:01.580191 8355 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:20:01.597529 8355 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:20:01.601586 8355 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:20:01.601878 8355 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:20:01.603016 8355 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:20:01.604439 8355 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:20:01.604544 8355 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:01.604691 8355 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:20:01.604763 8355 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:01.639420 8355 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:38419
I20260809 12:20:01.639465 8543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:38419 every 8 connection(s)
I20260809 12:20:01.640686 8355 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:20:01.647383 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 8355
I20260809 12:20:01.647588 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:20:01.648052 20191 external_mini_cluster.cc:1428] Setting key 6b31daf7a432b232933b5adc33f339b6
Aug 09 12:20:01 dist-test-slave-39l8 krb5kdc[7994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278001, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:20:01.662626 8547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:01.643167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:35587 (local address 127.19.183.254:45033)
0809 12:20:01.643674 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:01.643679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:01.644501 (+ 822us) server_negotiation.cc:407] Connection header received
0809 12:20:01.646343 (+ 1842us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:01.646348 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:01.646440 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:01.646563 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:01.648157 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:01.648902 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:01.651092 (+ 2190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:01.651838 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:01.652009 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:01.656071 (+ 4062us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:01.656076 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:01.656079 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:01.656112 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:01.659133 (+ 3021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:01.660148 (+ 1015us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:01.660151 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:01.660153 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:01.660274 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:01.660768 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:01.660771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:01.660773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:01.661027 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:01.661162 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:01.662193 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0809 12:20:01.662362 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":243,"threads_started":1}
I20260809 12:20:01.664049 8548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:01.643641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.194:35587)
0809 12:20:01.644341 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:01.644387 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:01.646073 (+ 1686us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:01.646865 (+ 792us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:01.646872 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:01.647360 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:20:01.647982 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:01.648000 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:01.649097 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:01.649103 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:01.650938 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:01.650953 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:01.652107 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:01.652113 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:01.652314 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:01.653020 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:20:01.653065 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:20:01.655817 (+ 2752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:20:01.659289 (+ 3472us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:01.659300 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:01.659318 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:01.660026 (+ 708us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:01.660385 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:01.660392 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:01.660398 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:01.660658 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:01.661182 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:20:01.661193 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:20:01.661677 (+ 484us) client_negotiation.cc:769] Sending connection context
0809 12:20:01.662015 (+ 338us) client_negotiation.cc:240] Negotiation successful
0809 12:20:01.662598 (+ 583us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"thread_start_us":276,"threads_started":1}
I20260809 12:20:01.665640 8545 heartbeater.cc:347] Connected to a master server at 127.19.183.254:45033
I20260809 12:20:01.665947 8545 heartbeater.cc:464] Registering TS with master...
I20260809 12:20:01.666747 8545 heartbeater.cc:511] Master 127.19.183.254:45033 requested a full tablet report, sending...
I20260809 12:20:01.668309 8065 ts_manager.cc:194] Registered new tserver with Master: 819e376295694741baea9fe748936978 (127.19.183.194:38419)
I20260809 12:20:01.669610 8065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:35587
WARNING: no policy specified for kudu/127.19.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
I20260809 12:20:01.718969 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:45033
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:46539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:20:02.062444 8552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:20:02.062816 8552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:20:02.062994 8552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:20:02.070214 8552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:20:02.070361 8552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:20:02.070540 8552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:20:02.080369 8552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:46539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:45033
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:20:02.081663 8552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:20:02.083015 8552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:20:02.096583 8560 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:02.096633 8559 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:02.096659 8562 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:20:02.097525 8552 server_base.cc:1034] running on GCE node
I20260809 12:20:02.098170 8552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:20:02.099157 8552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:20:02.100389 8552 hybrid_clock.cc:648] HybridClock initialized: now 1786278002100337 us; error 63 us; skew 500 ppm
I20260809 12:20:02.100697 8552 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:20:02 dist-test-slave-39l8 krb5kdc[7994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278002, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:20:02.103868 8552 init.cc:377] Logged in from keytab as kudu/127.19.183.195@KRBTEST.COM (short username kudu)
I20260809 12:20:02.105350 8552 webserver.cc:460] Webserver started at http://127.19.183.195:42115/ using document root <none> and password file <none>
I20260809 12:20:02.106035 8552 fs_manager.cc:359] Metadata directory not provided
I20260809 12:20:02.106191 8552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:20:02.106505 8552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:20:02.109469 8552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "41979697f97c4989bbf976682219d2c7"
format_stamp: "Formatted at 2026-08-09 12:20:02 on dist-test-slave-39l8"
server_key: "1198e40b482b45556af63f4451ad2e34"
server_key_iv: "6ea27b718a6303f5b1af1927cc64e2fb"
server_key_version: "encryptionkey@0"
I20260809 12:20:02.110327 8552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "41979697f97c4989bbf976682219d2c7"
format_stamp: "Formatted at 2026-08-09 12:20:02 on dist-test-slave-39l8"
server_key: "1198e40b482b45556af63f4451ad2e34"
server_key_iv: "6ea27b718a6303f5b1af1927cc64e2fb"
server_key_version: "encryptionkey@0"
I20260809 12:20:02.115701 8552 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260809 12:20:02.119163 8577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:02.120483 8552 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 12:20:02.120673 8552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "41979697f97c4989bbf976682219d2c7"
format_stamp: "Formatted at 2026-08-09 12:20:02 on dist-test-slave-39l8"
server_key: "1198e40b482b45556af63f4451ad2e34"
server_key_iv: "6ea27b718a6303f5b1af1927cc64e2fb"
server_key_version: "encryptionkey@0"
I20260809 12:20:02.120894 8552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:20:02.140813 8552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:20:02.144737 8552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:20:02.145047 8552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:20:02.146034 8552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:20:02.147415 8552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:20:02.147521 8552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:02.147677 8552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:20:02.147753 8552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:02.150588 8343 heartbeater.cc:503] Master 127.19.183.254:45033 was elected leader, sending a full tablet report...
I20260809 12:20:02.181771 8552 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:37603
I20260809 12:20:02.181820 8755 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:37603 every 8 connection(s)
I20260809 12:20:02.183100 8552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:20:02.186862 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 8552
I20260809 12:20:02.187054 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:20:02.187692 20191 external_mini_cluster.cc:1428] Setting key 3bb2ce2162016f7f40dc156e7b87041e
Aug 09 12:20:02 dist-test-slave-39l8 krb5kdc[7994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278002, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:20:02.203652 8758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:02.185730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:53219 (local address 127.19.183.254:45033)
0809 12:20:02.186170 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:02.186175 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:02.187294 (+ 1119us) server_negotiation.cc:407] Connection header received
0809 12:20:02.189756 (+ 2462us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:02.189761 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:02.189855 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:02.189962 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:02.191789 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.192584 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.194021 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.194405 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.194495 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:02.197706 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:02.197711 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:02.197714 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:02.197745 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:02.200300 (+ 2555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:02.200988 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:02.200992 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:02.200994 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:02.201132 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:02.201648 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:02.201653 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:02.201655 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:02.201976 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:02.202152 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:02.203061 (+ 909us) server_negotiation.cc:299] Negotiation successful
0809 12:20:02.203299 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":226,"threads_started":1}
I20260809 12:20:02.204540 8759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:02.186243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.195:53219)
0809 12:20:02.187094 (+ 851us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:02.187160 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:02.189419 (+ 2259us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:02.190233 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:02.190242 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:02.190883 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:20:02.191612 (+ 729us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:02.191631 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.192719 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.192722 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:02.193900 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:02.193912 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.194589 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.194594 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:02.194855 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:02.195452 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:20:02.195479 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:20:02.197530 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:20:02.200449 (+ 2919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:02.200455 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:02.200466 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:02.200883 (+ 417us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:02.201260 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:02.201264 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:02.201267 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:02.201506 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:02.202132 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:20:02.202140 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:20:02.202547 (+ 407us) client_negotiation.cc:769] Sending connection context
0809 12:20:02.202920 (+ 373us) client_negotiation.cc:240] Negotiation successful
0809 12:20:02.203391 (+ 471us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":559,"thread_start_us":293,"threads_started":1}
I20260809 12:20:02.205796 8756 heartbeater.cc:347] Connected to a master server at 127.19.183.254:45033
I20260809 12:20:02.206102 8756 heartbeater.cc:464] Registering TS with master...
I20260809 12:20:02.206877 8756 heartbeater.cc:511] Master 127.19.183.254:45033 requested a full tablet report, sending...
I20260809 12:20:02.208280 8065 ts_manager.cc:194] Registered new tserver with Master: 41979697f97c4989bbf976682219d2c7 (127.19.183.195:37603)
I20260809 12:20:02.209062 8065 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:53219
I20260809 12:20:02.215598 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:20:02.232398 8758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:02.220129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48236 (local address 127.19.183.254:45033)
0809 12:20:02.220349 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:02.220358 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:02.220749 (+ 391us) server_negotiation.cc:407] Connection header received
0809 12:20:02.220890 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:02.220894 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:02.220951 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:02.221026 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:02.222430 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.223181 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.224570 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.224954 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.225046 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:02.227553 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:02.227558 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:02.227561 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:02.227590 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:02.230134 (+ 2544us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:02.230820 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:02.230823 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:02.230825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:02.230938 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:02.231260 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:02.231263 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:02.231265 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:02.231555 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:02.231750 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:02.232019 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 12:20:02.232184 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 12:20:02.236456 8065 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48236:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:20:02.243546 8065 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:20:02.270959 8774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:02.255785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:37603 (local address 127.0.0.1:47408)
0809 12:20:02.256585 (+ 800us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:02.256641 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:02.256822 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:02.257903 (+ 1081us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:02.257911 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:02.257939 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:20:02.258374 (+ 435us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:02.258391 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.263919 (+ 5528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.263929 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:02.267146 (+ 3217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:02.267169 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.269935 (+ 2766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.269943 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:02.270198 (+ 255us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:02.270272 (+ 74us) client_negotiation.cc:769] Sending connection context
0809 12:20:02.270374 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 12:20:02.270577 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":550,"thread_start_us":405,"threads_started":1}
I20260809 12:20:02.271387 8776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:02.255961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47408 (local address 127.19.183.195:37603)
0809 12:20:02.257084 (+ 1123us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:02.257092 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:02.257131 (+ 39us) server_negotiation.cc:407] Connection header received
0809 12:20:02.257247 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:02.257256 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:02.257652 (+ 396us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:02.259937 (+ 2285us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:20:02.260324 (+ 387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.263722 (+ 3398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.268665 (+ 4943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.269664 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.270550 (+ 886us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:02.270758 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:02.270829 (+ 71us) server_negotiation.cc:299] Negotiation successful
0809 12:20:02.271017 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":915,"thread_start_us":739,"threads_started":1}
I20260809 12:20:02.275454 8653 tablet_service.cc:1467] Processing CreateTablet for tablet eb6a2237f20b48f7908a1feb3fb76e7b (DEFAULT_TABLE table=test-table [id=b5a7b503cde44f6592a020c8c07922e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:20:02.277014 8653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb6a2237f20b48f7908a1feb3fb76e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:20:02.277074 8782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:02.257740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43569 (local address 127.0.0.1:50616)
0809 12:20:02.267896 (+ 10156us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:02.267920 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:02.268070 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:02.269778 (+ 1708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:02.269785 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:02.269820 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:20:02.270269 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:02.270280 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.272286 (+ 2006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.272291 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:02.274443 (+ 2152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:02.274462 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.276231 (+ 1769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.276238 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:02.276409 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:02.276452 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 12:20:02.276593 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 12:20:02.276728 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9977,"thread_start_us":10425,"threads_started":1}
I20260809 12:20:02.277110 8783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:02.258432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50616 (local address 127.19.183.193:43569)
0809 12:20:02.269001 (+ 10569us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:02.269015 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:02.269055 (+ 40us) server_negotiation.cc:407] Connection header received
0809 12:20:02.269189 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:02.269202 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:02.269515 (+ 313us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:02.269771 (+ 256us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:20:02.270487 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.272138 (+ 1651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.274629 (+ 2491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.276012 (+ 1383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.276231 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:02.276410 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:02.276649 (+ 239us) server_negotiation.cc:299] Negotiation successful
0809 12:20:02.276800 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":10334,"thread_start_us":10117,"threads_started":1}
I20260809 12:20:02.279368 8779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:02.257088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:38419 (local address 127.0.0.1:45024)
0809 12:20:02.257678 (+ 590us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:02.257697 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:02.257814 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:02.271327 (+ 13513us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:02.271332 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:02.271365 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:20:02.271839 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:02.271852 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.273694 (+ 1842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.273702 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:02.275610 (+ 1908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:02.275637 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.277945 (+ 2308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.277952 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:02.278780 (+ 828us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:02.278814 (+ 34us) client_negotiation.cc:769] Sending connection context
0809 12:20:02.278913 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 12:20:02.279050 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":287,"threads_started":1}
I20260809 12:20:02.280126 8780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:02.257407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45024 (local address 127.19.183.194:38419)
0809 12:20:02.270444 (+ 13037us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:02.270452 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:02.270479 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:20:02.270574 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:02.270580 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:02.270831 (+ 251us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:02.271682 (+ 851us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:20:02.272017 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.273549 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.275804 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.277715 (+ 1911us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.277886 (+ 171us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:02.278046 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:02.279673 (+ 1627us) server_negotiation.cc:299] Negotiation successful
0809 12:20:02.279834 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12879,"thread_start_us":12775,"threads_started":1}
I20260809 12:20:02.280565 8251 tablet_service.cc:1467] Processing CreateTablet for tablet eb6a2237f20b48f7908a1feb3fb76e7b (DEFAULT_TABLE table=test-table [id=b5a7b503cde44f6592a020c8c07922e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:20:02.281725 8251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb6a2237f20b48f7908a1feb3fb76e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:20:02.283586 8445 tablet_service.cc:1467] Processing CreateTablet for tablet eb6a2237f20b48f7908a1feb3fb76e7b (DEFAULT_TABLE table=test-table [id=b5a7b503cde44f6592a020c8c07922e5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:20:02.284996 8445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eb6a2237f20b48f7908a1feb3fb76e7b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:20:02.292757 8784 tablet_bootstrap.cc:492] T eb6a2237f20b48f7908a1feb3fb76e7b P 41979697f97c4989bbf976682219d2c7: Bootstrap starting.
I20260809 12:20:02.294556 8785 tablet_bootstrap.cc:492] T eb6a2237f20b48f7908a1feb3fb76e7b P 4f3e4b638b13408a95ef66cdb91b2865: Bootstrap starting.
I20260809 12:20:02.297155 8784 tablet_bootstrap.cc:654] T eb6a2237f20b48f7908a1feb3fb76e7b P 41979697f97c4989bbf976682219d2c7: Neither blocks nor log segments found. Creating new log.
I20260809 12:20:02.298162 8784 log.cc:824] T eb6a2237f20b48f7908a1feb3fb76e7b P 41979697f97c4989bbf976682219d2c7: Log is configured to *not* fsync() on all Append() calls
I20260809 12:20:02.298266 8785 tablet_bootstrap.cc:654] T eb6a2237f20b48f7908a1feb3fb76e7b P 4f3e4b638b13408a95ef66cdb91b2865: Neither blocks nor log segments found. Creating new log.
I20260809 12:20:02.299587 8788 tablet_bootstrap.cc:492] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978: Bootstrap starting.
I20260809 12:20:02.299609 8785 log.cc:824] T eb6a2237f20b48f7908a1feb3fb76e7b P 4f3e4b638b13408a95ef66cdb91b2865: Log is configured to *not* fsync() on all Append() calls
I20260809 12:20:02.301375 8784 tablet_bootstrap.cc:492] T eb6a2237f20b48f7908a1feb3fb76e7b P 41979697f97c4989bbf976682219d2c7: No bootstrap required, opened a new log
I20260809 12:20:02.301666 8784 ts_tablet_manager.cc:1397] T eb6a2237f20b48f7908a1feb3fb76e7b P 41979697f97c4989bbf976682219d2c7: Time spent bootstrapping tablet: real 0.009s user 0.007s sys 0.000s
I20260809 12:20:02.302507 8785 tablet_bootstrap.cc:492] T eb6a2237f20b48f7908a1feb3fb76e7b P 4f3e4b638b13408a95ef66cdb91b2865: No bootstrap required, opened a new log
I20260809 12:20:02.302817 8785 ts_tablet_manager.cc:1397] T eb6a2237f20b48f7908a1feb3fb76e7b P 4f3e4b638b13408a95ef66cdb91b2865: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.003s
I20260809 12:20:02.302989 8788 tablet_bootstrap.cc:654] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978: Neither blocks nor log segments found. Creating new log.
I20260809 12:20:02.304220 8788 log.cc:824] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978: Log is configured to *not* fsync() on all Append() calls
I20260809 12:20:02.307166 8788 tablet_bootstrap.cc:492] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978: No bootstrap required, opened a new log
I20260809 12:20:02.307428 8788 ts_tablet_manager.cc:1397] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.001s
I20260809 12:20:02.310590 8784 raft_consensus.cc:348] T eb6a2237f20b48f7908a1feb3fb76e7b P 41979697f97c4989bbf976682219d2c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41979697f97c4989bbf976682219d2c7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 37603 } } peers { permanent_uuid: "819e376295694741baea9fe748936978" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38419 } } peers { permanent_uuid: "4f3e4b638b13408a95ef66cdb91b2865" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43569 } }
I20260809 12:20:02.311019 8784 raft_consensus.cc:374] T eb6a2237f20b48f7908a1feb3fb76e7b P 41979697f97c4989bbf976682219d2c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:20:02.311146 8784 raft_consensus.cc:729] T eb6a2237f20b48f7908a1feb3fb76e7b P 41979697f97c4989bbf976682219d2c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41979697f97c4989bbf976682219d2c7, State: Initialized, Role: FOLLOWER
I20260809 12:20:02.311755 8784 consensus_queue.cc:260] T eb6a2237f20b48f7908a1feb3fb76e7b P 41979697f97c4989bbf976682219d2c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41979697f97c4989bbf976682219d2c7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 37603 } } peers { permanent_uuid: "819e376295694741baea9fe748936978" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38419 } } peers { permanent_uuid: "4f3e4b638b13408a95ef66cdb91b2865" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43569 } }
I20260809 12:20:02.312873 8756 heartbeater.cc:503] Master 127.19.183.254:45033 was elected leader, sending a full tablet report...
I20260809 12:20:02.312789 8785 raft_consensus.cc:348] T eb6a2237f20b48f7908a1feb3fb76e7b P 4f3e4b638b13408a95ef66cdb91b2865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41979697f97c4989bbf976682219d2c7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 37603 } } peers { permanent_uuid: "819e376295694741baea9fe748936978" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38419 } } peers { permanent_uuid: "4f3e4b638b13408a95ef66cdb91b2865" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43569 } }
I20260809 12:20:02.313211 8785 raft_consensus.cc:374] T eb6a2237f20b48f7908a1feb3fb76e7b P 4f3e4b638b13408a95ef66cdb91b2865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:20:02.313299 8785 raft_consensus.cc:729] T eb6a2237f20b48f7908a1feb3fb76e7b P 4f3e4b638b13408a95ef66cdb91b2865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f3e4b638b13408a95ef66cdb91b2865, State: Initialized, Role: FOLLOWER
I20260809 12:20:02.313553 8784 ts_tablet_manager.cc:1428] T eb6a2237f20b48f7908a1feb3fb76e7b P 41979697f97c4989bbf976682219d2c7: Time spent starting tablet: real 0.012s user 0.011s sys 0.000s
I20260809 12:20:02.313984 8785 consensus_queue.cc:260] T eb6a2237f20b48f7908a1feb3fb76e7b P 4f3e4b638b13408a95ef66cdb91b2865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41979697f97c4989bbf976682219d2c7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 37603 } } peers { permanent_uuid: "819e376295694741baea9fe748936978" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38419 } } peers { permanent_uuid: "4f3e4b638b13408a95ef66cdb91b2865" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43569 } }
I20260809 12:20:02.315670 8785 ts_tablet_manager.cc:1428] T eb6a2237f20b48f7908a1feb3fb76e7b P 4f3e4b638b13408a95ef66cdb91b2865: Time spent starting tablet: real 0.013s user 0.011s sys 0.001s
I20260809 12:20:02.317934 8788 raft_consensus.cc:348] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41979697f97c4989bbf976682219d2c7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 37603 } } peers { permanent_uuid: "819e376295694741baea9fe748936978" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38419 } } peers { permanent_uuid: "4f3e4b638b13408a95ef66cdb91b2865" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43569 } }
I20260809 12:20:02.318342 8788 raft_consensus.cc:374] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:20:02.318444 8788 raft_consensus.cc:729] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 819e376295694741baea9fe748936978, State: Initialized, Role: FOLLOWER
I20260809 12:20:02.319008 8788 consensus_queue.cc:260] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41979697f97c4989bbf976682219d2c7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 37603 } } peers { permanent_uuid: "819e376295694741baea9fe748936978" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38419 } } peers { permanent_uuid: "4f3e4b638b13408a95ef66cdb91b2865" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43569 } }
I20260809 12:20:02.320686 8545 heartbeater.cc:503] Master 127.19.183.254:45033 was elected leader, sending a full tablet report...
I20260809 12:20:02.321135 8788 ts_tablet_manager.cc:1428] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978: Time spent starting tablet: real 0.014s user 0.010s sys 0.003s
I20260809 12:20:02.348783 8794 raft_consensus.cc:482] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:20:02.349136 8794 raft_consensus.cc:504] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41979697f97c4989bbf976682219d2c7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 37603 } } peers { permanent_uuid: "819e376295694741baea9fe748936978" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38419 } } peers { permanent_uuid: "4f3e4b638b13408a95ef66cdb91b2865" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43569 } }
I20260809 12:20:02.350895 8794 leader_election.cc:288] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 41979697f97c4989bbf976682219d2c7 (127.19.183.195:37603), 4f3e4b638b13408a95ef66cdb91b2865 (127.19.183.193:43569)
I20260809 12:20:02.359920 8776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:02.351414 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:52657 (local address 127.19.183.195:37603)
0809 12:20:02.351649 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:02.351657 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:02.352304 (+ 647us) server_negotiation.cc:407] Connection header received
0809 12:20:02.352547 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:02.352553 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:02.352633 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:02.352703 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:20:02.353541 (+ 838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.354492 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.357514 (+ 3022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.358789 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.358940 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:02.359011 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:02.359546 (+ 535us) server_negotiation.cc:299] Negotiation successful
0809 12:20:02.359670 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 12:20:02.359917 8783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:02.352446 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:50441 (local address 127.19.183.193:43569)
0809 12:20:02.352657 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:02.352665 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:02.352986 (+ 321us) server_negotiation.cc:407] Connection header received
0809 12:20:02.353129 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:02.353134 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:02.353210 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:02.353313 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:20:02.353969 (+ 656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.354667 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.357514 (+ 2847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.358789 (+ 1275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.358940 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:02.359011 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:02.359547 (+ 536us) server_negotiation.cc:299] Negotiation successful
0809 12:20:02.359673 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 12:20:02.360136 8796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:02.352231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43569 (local address 127.19.183.194:50441)
0809 12:20:02.352859 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:02.352873 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:02.352995 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:02.353349 (+ 354us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:02.353352 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:02.353371 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:20:02.353793 (+ 422us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:02.353806 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.354864 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.354874 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:02.357337 (+ 2463us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:02.357353 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.359059 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.359067 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:02.359333 (+ 266us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:02.359408 (+ 75us) client_negotiation.cc:769] Sending connection context
0809 12:20:02.359504 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 12:20:02.359680 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"mutex_wait_us":204,"thread_start_us":319,"threads_started":1}
I20260809 12:20:02.360508 8795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:02.351472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:37603 (local address 127.19.183.194:52657)
0809 12:20:02.352145 (+ 673us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:02.352191 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:02.352351 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:02.352884 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:02.352887 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:02.352909 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:20:02.353359 (+ 450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:02.353374 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.354643 (+ 1269us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.354655 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:02.357345 (+ 2690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:02.357353 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.359064 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.359069 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:02.359331 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:02.359406 (+ 75us) client_negotiation.cc:769] Sending connection context
0809 12:20:02.359504 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 12:20:02.359680 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":156,"thread_start_us":367,"threads_started":1}
I20260809 12:20:02.361043 8280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb6a2237f20b48f7908a1feb3fb76e7b" candidate_uuid: "819e376295694741baea9fe748936978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f3e4b638b13408a95ef66cdb91b2865" is_pre_election: true
I20260809 12:20:02.361373 8680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb6a2237f20b48f7908a1feb3fb76e7b" candidate_uuid: "819e376295694741baea9fe748936978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41979697f97c4989bbf976682219d2c7" is_pre_election: true
I20260809 12:20:02.361567 8280 raft_consensus.cc:2454] T eb6a2237f20b48f7908a1feb3fb76e7b P 4f3e4b638b13408a95ef66cdb91b2865 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 819e376295694741baea9fe748936978 in term 0.
I20260809 12:20:02.361799 8680 raft_consensus.cc:2454] T eb6a2237f20b48f7908a1feb3fb76e7b P 41979697f97c4989bbf976682219d2c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 819e376295694741baea9fe748936978 in term 0.
I20260809 12:20:02.362279 8379 leader_election.cc:302] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f3e4b638b13408a95ef66cdb91b2865, 819e376295694741baea9fe748936978; no voters:
I20260809 12:20:02.362644 8794 raft_consensus.cc:2781] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:20:02.362816 8794 raft_consensus.cc:482] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:20:02.362943 8794 raft_consensus.cc:3037] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:20:02.365877 8794 raft_consensus.cc:504] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41979697f97c4989bbf976682219d2c7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 37603 } } peers { permanent_uuid: "819e376295694741baea9fe748936978" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38419 } } peers { permanent_uuid: "4f3e4b638b13408a95ef66cdb91b2865" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43569 } }
I20260809 12:20:02.366529 8794 leader_election.cc:288] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978 [CANDIDATE]: Term 1 election: Requested vote from peers 41979697f97c4989bbf976682219d2c7 (127.19.183.195:37603), 4f3e4b638b13408a95ef66cdb91b2865 (127.19.183.193:43569)
I20260809 12:20:02.366918 8680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb6a2237f20b48f7908a1feb3fb76e7b" candidate_uuid: "819e376295694741baea9fe748936978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41979697f97c4989bbf976682219d2c7"
I20260809 12:20:02.367012 8280 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eb6a2237f20b48f7908a1feb3fb76e7b" candidate_uuid: "819e376295694741baea9fe748936978" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4f3e4b638b13408a95ef66cdb91b2865"
I20260809 12:20:02.367120 8680 raft_consensus.cc:3037] T eb6a2237f20b48f7908a1feb3fb76e7b P 41979697f97c4989bbf976682219d2c7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:20:02.367254 8280 raft_consensus.cc:3037] T eb6a2237f20b48f7908a1feb3fb76e7b P 4f3e4b638b13408a95ef66cdb91b2865 [term 0 FOLLOWER]: Advancing to term 1
W20260809 12:20:02.368299 8344 tablet.cc:2369] T eb6a2237f20b48f7908a1feb3fb76e7b P 4f3e4b638b13408a95ef66cdb91b2865: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:20:02.369486 8680 raft_consensus.cc:2454] T eb6a2237f20b48f7908a1feb3fb76e7b P 41979697f97c4989bbf976682219d2c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 819e376295694741baea9fe748936978 in term 1.
I20260809 12:20:02.369886 8379 leader_election.cc:302] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 41979697f97c4989bbf976682219d2c7, 819e376295694741baea9fe748936978; no voters:
I20260809 12:20:02.370136 8280 raft_consensus.cc:2454] T eb6a2237f20b48f7908a1feb3fb76e7b P 4f3e4b638b13408a95ef66cdb91b2865 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 819e376295694741baea9fe748936978 in term 1.
I20260809 12:20:02.370178 8794 raft_consensus.cc:2781] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:20:02.370683 8794 raft_consensus.cc:686] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978 [term 1 LEADER]: Becoming Leader. State: Replica: 819e376295694741baea9fe748936978, State: Running, Role: LEADER
I20260809 12:20:02.371203 8794 consensus_queue.cc:237] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41979697f97c4989bbf976682219d2c7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 37603 } } peers { permanent_uuid: "819e376295694741baea9fe748936978" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38419 } } peers { permanent_uuid: "4f3e4b638b13408a95ef66cdb91b2865" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43569 } }
I20260809 12:20:02.375646 8065 catalog_manager.cc:5495] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978 reported cstate change: term changed from 0 to 1, leader changed from <none> to 819e376295694741baea9fe748936978 (127.19.183.194). New cstate: current_term: 1 leader_uuid: "819e376295694741baea9fe748936978" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "41979697f97c4989bbf976682219d2c7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 37603 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "819e376295694741baea9fe748936978" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4f3e4b638b13408a95ef66cdb91b2865" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43569 } health_report { overall_health: UNKNOWN } } }
I20260809 12:20:02.391269 8780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:02.386114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45040 (local address 127.19.183.194:38419)
0809 12:20:02.386293 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:02.386298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:02.386351 (+ 53us) server_negotiation.cc:407] Connection header received
0809 12:20:02.386540 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:02.386543 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:02.386616 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:02.386809 (+ 193us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:20:02.387154 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.387843 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.389252 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.389662 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.389771 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:02.390212 (+ 441us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:20:02.390698 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:20:02.390907 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:02.390980 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 12:20:02.391051 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
W20260809 12:20:02.392692 8546 tablet.cc:2369] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:20:02.399395 8280 raft_consensus.cc:1261] T eb6a2237f20b48f7908a1feb3fb76e7b P 4f3e4b638b13408a95ef66cdb91b2865 [term 1 FOLLOWER]: Refusing update from remote peer 819e376295694741baea9fe748936978: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:20:02.399670 8680 raft_consensus.cc:1261] T eb6a2237f20b48f7908a1feb3fb76e7b P 41979697f97c4989bbf976682219d2c7 [term 1 FOLLOWER]: Refusing update from remote peer 819e376295694741baea9fe748936978: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:20:02.400385 8797 consensus_queue.cc:1035] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4f3e4b638b13408a95ef66cdb91b2865" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43569 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:20:02.400780 8794 consensus_queue.cc:1035] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41979697f97c4989bbf976682219d2c7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 37603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:20:02.426538 8060 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:20:02.426901 8060 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48236:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:20:02.431548 8060 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0: Sending DeleteTablet for 3 replicas of tablet eb6a2237f20b48f7908a1feb3fb76e7b
I20260809 12:20:02.432730 8653 tablet_service.cc:1514] Processing DeleteTablet for tablet eb6a2237f20b48f7908a1feb3fb76e7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:20:02 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:47408
I20260809 12:20:02.432947 8251 tablet_service.cc:1514] Processing DeleteTablet for tablet eb6a2237f20b48f7908a1feb3fb76e7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:20:02 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:50616
I20260809 12:20:02.433116 8445 tablet_service.cc:1514] Processing DeleteTablet for tablet eb6a2237f20b48f7908a1feb3fb76e7b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:20:02 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:45024
I20260809 12:20:02.433705 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 8151
I20260809 12:20:02.462353 8820 tablet_replica.cc:330] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978: stopping tablet replica
I20260809 12:20:02.463065 8820 raft_consensus.cc:2229] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:20:02.463961 8820 raft_consensus.cc:2258] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:20:02.465437 8815 tablet_replica.cc:330] T eb6a2237f20b48f7908a1feb3fb76e7b P 41979697f97c4989bbf976682219d2c7: stopping tablet replica
I20260809 12:20:02.465991 8815 raft_consensus.cc:2229] T eb6a2237f20b48f7908a1feb3fb76e7b P 41979697f97c4989bbf976682219d2c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:20:02.466619 8815 raft_consensus.cc:2258] T eb6a2237f20b48f7908a1feb3fb76e7b P 41979697f97c4989bbf976682219d2c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:20:02.471422 8816 tablet_replica.cc:330] T eb6a2237f20b48f7908a1feb3fb76e7b P 4f3e4b638b13408a95ef66cdb91b2865: stopping tablet replica
I20260809 12:20:02.471843 8816 raft_consensus.cc:2229] T eb6a2237f20b48f7908a1feb3fb76e7b P 4f3e4b638b13408a95ef66cdb91b2865 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:20:02.472385 8816 raft_consensus.cc:2258] T eb6a2237f20b48f7908a1feb3fb76e7b P 4f3e4b638b13408a95ef66cdb91b2865 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:20:02.472666 8820 ts_tablet_manager.cc:1905] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:20:02.472712 8815 ts_tablet_manager.cc:1905] T eb6a2237f20b48f7908a1feb3fb76e7b P 41979697f97c4989bbf976682219d2c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:20:02.474522 8816 ts_tablet_manager.cc:1905] T eb6a2237f20b48f7908a1feb3fb76e7b P 4f3e4b638b13408a95ef66cdb91b2865: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:20:02.481743 8816 ts_tablet_manager.cc:1918] T eb6a2237f20b48f7908a1feb3fb76e7b P 4f3e4b638b13408a95ef66cdb91b2865: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:20:02.481737 8815 ts_tablet_manager.cc:1918] T eb6a2237f20b48f7908a1feb3fb76e7b P 41979697f97c4989bbf976682219d2c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:20:02.481909 8815 log.cc:1196] T eb6a2237f20b48f7908a1feb3fb76e7b P 41979697f97c4989bbf976682219d2c7: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/eb6a2237f20b48f7908a1feb3fb76e7b
I20260809 12:20:02.482062 8816 log.cc:1196] T eb6a2237f20b48f7908a1feb3fb76e7b P 4f3e4b638b13408a95ef66cdb91b2865: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/eb6a2237f20b48f7908a1feb3fb76e7b
I20260809 12:20:02.482476 8815 ts_tablet_manager.cc:1939] T eb6a2237f20b48f7908a1feb3fb76e7b P 41979697f97c4989bbf976682219d2c7: Deleting consensus metadata
I20260809 12:20:02.482829 8816 ts_tablet_manager.cc:1939] T eb6a2237f20b48f7908a1feb3fb76e7b P 4f3e4b638b13408a95ef66cdb91b2865: Deleting consensus metadata
Aug 09 12:20:02 dist-test-slave-39l8 krb5kdc[7994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.193@KRBTEST.COM
I20260809 12:20:02.484524 8043 catalog_manager.cc:4841] TS 41979697f97c4989bbf976682219d2c7 (127.19.183.195:37603): tablet eb6a2237f20b48f7908a1feb3fb76e7b (table test-table [id=b5a7b503cde44f6592a020c8c07922e5]) successfully deleted
I20260809 12:20:02.484977 8043 catalog_manager.cc:4841] TS 4f3e4b638b13408a95ef66cdb91b2865 (127.19.183.193:43569): tablet eb6a2237f20b48f7908a1feb3fb76e7b (table test-table [id=b5a7b503cde44f6592a020c8c07922e5]) successfully deleted
I20260809 12:20:02.488588 8820 ts_tablet_manager.cc:1918] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:20:02.488759 8820 log.cc:1196] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/eb6a2237f20b48f7908a1feb3fb76e7b
I20260809 12:20:02.489331 8820 ts_tablet_manager.cc:1939] T eb6a2237f20b48f7908a1feb3fb76e7b P 819e376295694741baea9fe748936978: Deleting consensus metadata
I20260809 12:20:02.491257 8043 catalog_manager.cc:4841] TS 819e376295694741baea9fe748936978 (127.19.183.194:38419): tablet eb6a2237f20b48f7908a1feb3fb76e7b (table test-table [id=b5a7b503cde44f6592a020c8c07922e5]) successfully deleted
I20260809 12:20:02.494585 8783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:02.435338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50626 (local address 127.19.183.193:43569)
0809 12:20:02.435571 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:02.435580 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:02.458219 (+ 22639us) server_negotiation.cc:407] Connection header received
0809 12:20:02.458597 (+ 378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:02.458607 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:02.458755 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:02.458869 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:02.467206 (+ 8337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.468890 (+ 1684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.478939 (+ 10049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.479678 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.479787 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:02.486375 (+ 6588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:02.486392 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:02.486411 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:02.486455 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:02.490629 (+ 4174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:02.492211 (+ 1582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:02.492222 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:02.492226 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:02.492387 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:02.492857 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:02.492862 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:02.492866 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:02.493333 (+ 467us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:02.493651 (+ 318us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:02.493992 (+ 341us) server_negotiation.cc:299] Negotiation successful
0809 12:20:02.494218 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 12:20:02.495375 8338 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:20:02.587646 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 8151
I20260809 12:20:02.621344 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 8355
Aug 09 12:20:02 dist-test-slave-39l8 krb5kdc[7994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.194@KRBTEST.COM
I20260809 12:20:02.646469 8780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:02.622329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45054 (local address 127.19.183.194:38419)
0809 12:20:02.622510 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:02.622514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:02.622534 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:20:02.622604 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:02.622607 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:02.622668 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:02.622805 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:02.624501 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.625469 (+ 968us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.627540 (+ 2071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.628044 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.628216 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:02.638680 (+ 10464us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:02.638692 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:02.638722 (+ 30us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:02.638775 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:02.643079 (+ 4304us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:02.643935 (+ 856us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:02.643945 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:02.643949 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:02.644118 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:02.644520 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:02.644524 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:02.644528 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:02.644994 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:02.645341 (+ 347us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:02.645637 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 12:20:02.645877 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:20:02.647315 8533 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:20:02.760457 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 8355
I20260809 12:20:02.785146 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 8552
Aug 09 12:20:02 dist-test-slave-39l8 krb5kdc[7994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786277999, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.195@KRBTEST.COM
I20260809 12:20:02.799854 8776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:02.785936 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47414 (local address 127.19.183.195:37603)
0809 12:20:02.786128 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:02.786136 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:02.786201 (+ 65us) server_negotiation.cc:407] Connection header received
0809 12:20:02.786428 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:02.786431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:02.786496 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:02.786614 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:02.787832 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.788528 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.789757 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:02.790134 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:02.790270 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:02.794805 (+ 4535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:02.794820 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:02.794854 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:02.794895 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:02.797363 (+ 2468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:02.797961 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:02.797967 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:02.797969 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:02.798090 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:02.798439 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:02.798442 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:02.798444 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:02.798836 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:02.799059 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:02.799318 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 12:20:02.799481 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 12:20:02.800544 8749 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:20:02.889809 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 8552
I20260809 12:20:02.915344 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 8019
I20260809 12:20:02.916563 8107 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:20:03.026489 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 8019
I20260809 12:20:03.060420 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:45033
--webserver_interface=127.19.183.254
--webserver_port=35351
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:46539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:45033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:20:03.412460 8830 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:20:03.412770 8830 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:20:03.412860 8830 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:20:03.420310 8830 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:20:03.420415 8830 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:20:03.420460 8830 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:20:03.420517 8830 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:20:03.420563 8830 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:20:03.430418 8830 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:46539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:45033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:45033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=35351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:20:03.432228 8830 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:20:03.433840 8830 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:20:03.461256 8839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:03.461310 8838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:03.461275 8842 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:20:03.462394 8830 server_base.cc:1034] running on GCE node
I20260809 12:20:03.463325 8830 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:20:03.464877 8830 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:20:03.466156 8830 hybrid_clock.cc:648] HybridClock initialized: now 1786278003466110 us; error 75 us; skew 500 ppm
I20260809 12:20:03.466580 8830 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:20:03 dist-test-slave-39l8 krb5kdc[7994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278003, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:20:03.470089 8830 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:20:03.471791 8830 webserver.cc:460] Webserver started at http://127.19.183.254:35351/ using document root <none> and password file <none>
I20260809 12:20:03.472591 8830 fs_manager.cc:359] Metadata directory not provided
I20260809 12:20:03.472769 8830 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:20:03.478880 8830 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260809 12:20:03.481876 8851 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:03.483274 8830 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:20:03.483469 8830 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "c8a0592aa088438ea86a0bbb545d93c0"
format_stamp: "Formatted at 2026-08-09 12:20:00 on dist-test-slave-39l8"
server_key: "bdf58f8d2789001a231b04dfb4fc1d30"
server_key_iv: "04357a4fa0b65ba1a45d16c7cbaf5c2a"
server_key_version: "encryptionkey@0"
I20260809 12:20:03.484546 8830 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:20:03.503019 8830 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:20:03.506943 8830 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:20:03.507217 8830 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:20:03.526495 8830 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:45033
I20260809 12:20:03.526515 8935 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:45033 every 8 connection(s)
I20260809 12:20:03.528017 8830 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:20:03.531747 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 8830
I20260809 12:20:03.532603 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:43569
--local_ip_for_outbound_sockets=127.19.183.193
--tserver_master_addrs=127.19.183.254:45033
--webserver_port=42227
--webserver_interface=127.19.183.193
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:46539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:20:03.535020 8939 sys_catalog.cc:263] Verifying existing consensus state
I20260809 12:20:03.538161 8939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0: Bootstrap starting.
I20260809 12:20:03.557634 8939 log.cc:824] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0: Log is configured to *not* fsync() on all Append() calls
I20260809 12:20:03.570159 8939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:20:03.570740 8939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0: Bootstrap complete.
I20260809 12:20:03.580566 8939 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0592aa088438ea86a0bbb545d93c0" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45033 } }
I20260809 12:20:03.581424 8939 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8a0592aa088438ea86a0bbb545d93c0, State: Initialized, Role: FOLLOWER
I20260809 12:20:03.582019 8939 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0592aa088438ea86a0bbb545d93c0" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45033 } }
I20260809 12:20:03.582267 8939 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:20:03.582399 8939 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:20:03.582574 8939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 1 FOLLOWER]: Advancing to term 2
I20260809 12:20:03.586287 8939 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0592aa088438ea86a0bbb545d93c0" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45033 } }
I20260809 12:20:03.586853 8939 leader_election.cc:302] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8a0592aa088438ea86a0bbb545d93c0; no voters:
I20260809 12:20:03.587531 8939 leader_election.cc:288] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 12:20:03.587716 8943 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 12:20:03.588969 8943 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 2 LEADER]: Becoming Leader. State: Replica: c8a0592aa088438ea86a0bbb545d93c0, State: Running, Role: LEADER
I20260809 12:20:03.589515 8943 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0592aa088438ea86a0bbb545d93c0" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45033 } }
I20260809 12:20:03.589952 8939 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:20:03.592267 8945 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "c8a0592aa088438ea86a0bbb545d93c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0592aa088438ea86a0bbb545d93c0" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45033 } } }
I20260809 12:20:03.592711 8945 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [sys.catalog]: This master's current role is: LEADER
I20260809 12:20:03.594462 8951 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:20:03.594372 8947 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8a0592aa088438ea86a0bbb545d93c0. Latest consensus state: current_term: 2 leader_uuid: "c8a0592aa088438ea86a0bbb545d93c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0592aa088438ea86a0bbb545d93c0" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45033 } } }
I20260809 12:20:03.594785 8947 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [sys.catalog]: This master's current role is: LEADER
I20260809 12:20:03.602792 8951 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:20:03.604238 8951 catalog_manager.cc:1259] Loaded cluster ID: 54a46fa9988e4b1896acc48c1d2a09d1
I20260809 12:20:03.604394 8951 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:20:03.608450 8951 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 12:20:03.634457 8951 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 12:20:03.655020 8951 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:20:03.656795 8951 catalog_manager.cc:5879] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0: Loaded TSK: 0
I20260809 12:20:03.658280 8951 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 12:20:03.895737 8941 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:20:03.896010 8941 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:20:03.896184 8941 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:20:03.903497 8941 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:20:03.903597 8941 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:20:03.903795 8941 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:20:03.913245 8941 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:46539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:43569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=42227
--tserver_master_addrs=127.19.183.254:45033
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:20:03.914530 8941 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:20:03.915917 8941 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:20:03.929255 8977 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:03.929322 8976 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:03.931617 8984 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:20:03.932438 8941 server_base.cc:1034] running on GCE node
I20260809 12:20:03.933115 8941 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:20:03.934266 8941 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:20:03.935503 8941 hybrid_clock.cc:648] HybridClock initialized: now 1786278003935451 us; error 37 us; skew 500 ppm
I20260809 12:20:03.935869 8941 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:20:03 dist-test-slave-39l8 krb5kdc[7994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278003, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:20:03.943806 8941 init.cc:377] Logged in from keytab as kudu/127.19.183.193@KRBTEST.COM (short username kudu)
I20260809 12:20:03.945672 8941 webserver.cc:460] Webserver started at http://127.19.183.193:42227/ using document root <none> and password file <none>
I20260809 12:20:03.946592 8941 fs_manager.cc:359] Metadata directory not provided
I20260809 12:20:03.946772 8941 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:20:03.953809 8941 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.002s
I20260809 12:20:03.957185 8996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:03.958781 8941 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:20:03.958938 8941 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "4f3e4b638b13408a95ef66cdb91b2865"
format_stamp: "Formatted at 2026-08-09 12:20:01 on dist-test-slave-39l8"
server_key: "60ccc26f4fe3ebb0d68f6e558d5f911c"
server_key_iv: "277f1781e4341768f7e8f7490ac58b51"
server_key_version: "encryptionkey@0"
I20260809 12:20:03.960297 8941 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:20:03.974037 8941 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:20:03.978826 8941 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:20:03.979096 8941 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:20:03.980219 8941 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:20:03.981693 8941 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:20:03.981777 8941 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:03.981848 8941 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:20:03.981902 8941 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:04.017189 8941 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:43569
I20260809 12:20:04.017370 9189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:43569 every 8 connection(s)
I20260809 12:20:04.019330 8941 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:20:04.027410 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 8941
I20260809 12:20:04.028065 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:38419
--local_ip_for_outbound_sockets=127.19.183.194
--tserver_master_addrs=127.19.183.254:45033
--webserver_port=36611
--webserver_interface=127.19.183.194
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:46539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:20:04 dist-test-slave-39l8 krb5kdc[7994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278003, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:20:04.045070 9196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:04.023311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:55629 (local address 127.19.183.254:45033)
0809 12:20:04.023939 (+ 628us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:04.023953 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:04.024047 (+ 94us) server_negotiation.cc:407] Connection header received
0809 12:20:04.026136 (+ 2089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:04.026148 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:04.026435 (+ 287us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:04.026785 (+ 350us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:04.028337 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:04.030346 (+ 2009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:04.031889 (+ 1543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:04.032965 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:04.033186 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:04.036529 (+ 3343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:04.036541 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:04.036561 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:04.036628 (+ 67us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:04.040574 (+ 3946us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:04.041320 (+ 746us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:04.041329 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:04.041336 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:04.041514 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:04.042010 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:04.042015 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:04.042018 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:04.042464 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:04.042682 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:04.043484 (+ 802us) server_negotiation.cc:299] Negotiation successful
0809 12:20:04.043877 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":415,"thread_start_us":272,"threads_started":1}
I20260809 12:20:04.045069 9193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:04.023104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.193:55629)
0809 12:20:04.023917 (+ 813us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:04.023953 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:04.025620 (+ 1667us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:04.026913 (+ 1293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:04.026919 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:04.027507 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:20:04.028065 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:04.028082 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:04.030506 (+ 2424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:04.030510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:04.031740 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:04.031753 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:04.033159 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:04.033164 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:04.033352 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:04.033985 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:20:04.034005 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:20:04.036245 (+ 2240us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:20:04.040731 (+ 4486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:04.040739 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:04.040753 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:04.041175 (+ 422us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:04.041644 (+ 469us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:04.041648 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:04.041652 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:04.041877 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:04.042616 (+ 739us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:20:04.042624 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:20:04.042982 (+ 358us) client_negotiation.cc:769] Sending connection context
0809 12:20:04.043250 (+ 268us) client_negotiation.cc:240] Negotiation successful
0809 12:20:04.043770 (+ 520us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":528,"thread_start_us":275,"threads_started":1}
I20260809 12:20:04.048462 9190 heartbeater.cc:347] Connected to a master server at 127.19.183.254:45033
I20260809 12:20:04.048780 9190 heartbeater.cc:464] Registering TS with master...
I20260809 12:20:04.049695 9190 heartbeater.cc:511] Master 127.19.183.254:45033 requested a full tablet report, sending...
I20260809 12:20:04.051940 8879 ts_manager.cc:194] Registered new tserver with Master: 4f3e4b638b13408a95ef66cdb91b2865 (127.19.183.193:43569)
I20260809 12:20:04.053431 8879 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:55629
W20260809 12:20:04.370110 9197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:20:04.370363 9197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:20:04.370517 9197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:20:04.377321 9197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:20:04.377429 9197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:20:04.377570 9197 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:20:04.386236 9197 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:46539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:38419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=36611
--tserver_master_addrs=127.19.183.254:45033
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:20:04.387315 9197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:20:04.388610 9197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:20:04.401453 9208 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:04.401480 9207 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:04.402038 9210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:20:04.402127 9197 server_base.cc:1034] running on GCE node
I20260809 12:20:04.402748 9197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:20:04.403573 9197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:20:04.404793 9197 hybrid_clock.cc:648] HybridClock initialized: now 1786278004404755 us; error 47 us; skew 500 ppm
I20260809 12:20:04.405109 9197 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:20:04 dist-test-slave-39l8 krb5kdc[7994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278004, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:20:04.408301 9197 init.cc:377] Logged in from keytab as kudu/127.19.183.194@KRBTEST.COM (short username kudu)
I20260809 12:20:04.409780 9197 webserver.cc:460] Webserver started at http://127.19.183.194:36611/ using document root <none> and password file <none>
I20260809 12:20:04.410434 9197 fs_manager.cc:359] Metadata directory not provided
I20260809 12:20:04.410593 9197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:20:04.415817 9197 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:20:04.418524 9218 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:04.419947 9197 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:20:04.420131 9197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "819e376295694741baea9fe748936978"
format_stamp: "Formatted at 2026-08-09 12:20:01 on dist-test-slave-39l8"
server_key: "411bf0dd8e189818b91170f619d9139c"
server_key_iv: "cea7bb78442f32bdd02a4805ca12cb0b"
server_key_version: "encryptionkey@0"
I20260809 12:20:04.421078 9197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:20:04.449652 9197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:20:04.453637 9197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:20:04.453910 9197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:20:04.454937 9197 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:20:04.456347 9197 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:20:04.456468 9197 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:04.456607 9197 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:20:04.456720 9197 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:04.491739 9197 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:38419
I20260809 12:20:04.491825 9385 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:38419 every 8 connection(s)
I20260809 12:20:04.493232 9197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:20:04.500571 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 9197
I20260809 12:20:04.501191 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:37603
--local_ip_for_outbound_sockets=127.19.183.195
--tserver_master_addrs=127.19.183.254:45033
--webserver_port=42115
--webserver_interface=127.19.183.195
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:46539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:20:04 dist-test-slave-39l8 krb5kdc[7994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278004, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:20:04.513099 9196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:04.495839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:45653 (local address 127.19.183.254:45033)
0809 12:20:04.496024 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:04.496028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:04.497216 (+ 1188us) server_negotiation.cc:407] Connection header received
0809 12:20:04.499325 (+ 2109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:04.499331 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:04.499418 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:04.499529 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:04.501069 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:04.501855 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:04.503512 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:04.503938 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:04.504072 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:04.507437 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:04.507441 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:04.507445 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:04.507476 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:04.510011 (+ 2535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:04.510799 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:04.510803 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:04.510805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:04.510942 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:04.511400 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:04.511404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:04.511406 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:04.511671 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:04.511836 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:04.512536 (+ 700us) server_negotiation.cc:299] Negotiation successful
0809 12:20:04.512773 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:20:04.514047 9388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:04.496308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.194:45653)
0809 12:20:04.497040 (+ 732us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:04.497103 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:04.499034 (+ 1931us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:04.499772 (+ 738us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:04.499780 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:04.500298 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:20:04.500902 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:04.500920 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:04.502022 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:04.502027 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:04.503354 (+ 1327us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:04.503367 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:04.504170 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:04.504178 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:04.504399 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:04.505072 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:20:04.505093 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:20:04.507228 (+ 2135us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:20:04.510172 (+ 2944us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:04.510181 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:04.510196 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:04.510627 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:04.511066 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:04.511071 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:04.511074 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:04.511264 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:04.511825 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:20:04.511832 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:20:04.512157 (+ 325us) client_negotiation.cc:769] Sending connection context
0809 12:20:04.512414 (+ 257us) client_negotiation.cc:240] Negotiation successful
0809 12:20:04.512921 (+ 507us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":451,"thread_start_us":269,"threads_started":1}
I20260809 12:20:04.515569 9386 heartbeater.cc:347] Connected to a master server at 127.19.183.254:45033
I20260809 12:20:04.515875 9386 heartbeater.cc:464] Registering TS with master...
I20260809 12:20:04.516608 9386 heartbeater.cc:511] Master 127.19.183.254:45033 requested a full tablet report, sending...
I20260809 12:20:04.518267 8879 ts_manager.cc:194] Registered new tserver with Master: 819e376295694741baea9fe748936978 (127.19.183.194:38419)
I20260809 12:20:04.519157 8879 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:45653
W20260809 12:20:04.846613 9389 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:20:04.846982 9389 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:20:04.847169 9389 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:20:04.854297 9389 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:20:04.854398 9389 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:20:04.854563 9389 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:20:04.864354 9389 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:46539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:37603
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=42115
--tserver_master_addrs=127.19.183.254:45033
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:20:04.865708 9389 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:20:04.867054 9389 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:20:04.880498 9394 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:04.880545 9398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:04.880515 9395 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:20:04.882018 9389 server_base.cc:1034] running on GCE node
I20260809 12:20:04.882547 9389 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:20:04.883498 9389 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:20:04.884784 9389 hybrid_clock.cc:648] HybridClock initialized: now 1786278004884715 us; error 47 us; skew 500 ppm
I20260809 12:20:04.885156 9389 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:20:04 dist-test-slave-39l8 krb5kdc[7994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278004, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:20:04.888501 9389 init.cc:377] Logged in from keytab as kudu/127.19.183.195@KRBTEST.COM (short username kudu)
I20260809 12:20:04.889997 9389 webserver.cc:460] Webserver started at http://127.19.183.195:42115/ using document root <none> and password file <none>
I20260809 12:20:04.890812 9389 fs_manager.cc:359] Metadata directory not provided
I20260809 12:20:04.890974 9389 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:20:04.896505 9389 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:20:04.899259 9406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:04.900484 9389 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 12:20:04.900663 9389 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "41979697f97c4989bbf976682219d2c7"
format_stamp: "Formatted at 2026-08-09 12:20:02 on dist-test-slave-39l8"
server_key: "1198e40b482b45556af63f4451ad2e34"
server_key_iv: "6ea27b718a6303f5b1af1927cc64e2fb"
server_key_version: "encryptionkey@0"
I20260809 12:20:04.901645 9389 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:20:04.934089 9389 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:20:04.941002 9389 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:20:04.941419 9389 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:20:04.942847 9389 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:20:04.944983 9389 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:20:04.945129 9389 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:04.945262 9389 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:20:04.945360 9389 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:04.983650 9389 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:37603
I20260809 12:20:04.983737 9569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:37603 every 8 connection(s)
I20260809 12:20:04.985184 9389 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:20:04.993001 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 9389
Aug 09 12:20:04 dist-test-slave-39l8 krb5kdc[7994](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278004, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:20:05.007834 9196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:04.987991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:46359 (local address 127.19.183.254:45033)
0809 12:20:04.988169 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:04.988173 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:04.989387 (+ 1214us) server_negotiation.cc:407] Connection header received
0809 12:20:04.991324 (+ 1937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:04.991328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:04.991396 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:04.991539 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:04.993645 (+ 2106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:04.994644 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:04.996268 (+ 1624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:04.997057 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:04.997236 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:05.001167 (+ 3931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:05.001172 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:05.001175 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:05.001201 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:05.004244 (+ 3043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:05.005255 (+ 1011us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:05.005264 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:05.005269 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:05.005447 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:05.005954 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:05.005959 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:05.005963 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:05.006320 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:05.006498 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:05.007392 (+ 894us) server_negotiation.cc:299] Negotiation successful
0809 12:20:05.007549 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 12:20:05.009425 9573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:04.988464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.195:46359)
0809 12:20:04.989249 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:04.989286 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:04.991054 (+ 1768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:04.991885 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:04.991899 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:04.992535 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:20:04.993444 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:04.993473 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:04.994840 (+ 1367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:04.994846 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:04.996084 (+ 1238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:04.996131 (+ 47us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:04.997340 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:04.997347 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:04.997617 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:04.998320 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:20:04.998344 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:20:05.000951 (+ 2607us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:20:05.004452 (+ 3501us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:05.004462 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:05.004478 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:05.005099 (+ 621us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:05.005585 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:05.005591 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:05.005615 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:05.005817 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:05.006477 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:20:05.006494 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:20:05.006947 (+ 453us) client_negotiation.cc:769] Sending connection context
0809 12:20:05.007315 (+ 368us) client_negotiation.cc:240] Negotiation successful
0809 12:20:05.007971 (+ 656us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"thread_start_us":330,"threads_started":1}
I20260809 12:20:05.010093 9576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:04.994004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48242 (local address 127.19.183.254:45033)
0809 12:20:04.994672 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:04.994681 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:04.995237 (+ 556us) server_negotiation.cc:407] Connection header received
0809 12:20:04.995445 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:04.995450 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:04.995522 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:04.995643 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:04.997685 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:04.998422 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.000112 (+ 1690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.000619 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.000717 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:05.003644 (+ 2927us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:05.003652 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:05.003657 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:05.003714 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:05.006941 (+ 3227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:05.007872 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:05.007881 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:05.007886 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:05.008038 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:05.008619 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:05.008622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:05.008624 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:05.008861 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:05.009052 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:05.009458 (+ 406us) server_negotiation.cc:299] Negotiation successful
0809 12:20:05.009716 (+ 258us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":500,"thread_start_us":314,"threads_started":1}
I20260809 12:20:05.011241 9571 heartbeater.cc:347] Connected to a master server at 127.19.183.254:45033
I20260809 12:20:05.011584 9571 heartbeater.cc:464] Registering TS with master...
I20260809 12:20:05.012537 9571 heartbeater.cc:511] Master 127.19.183.254:45033 requested a full tablet report, sending...
I20260809 12:20:05.014048 8876 ts_manager.cc:194] Registered new tserver with Master: 41979697f97c4989bbf976682219d2c7 (127.19.183.195:37603)
I20260809 12:20:05.015066 8876 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:46359
I20260809 12:20:05.025070 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:20:05.032032 9576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:05.025926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48248 (local address 127.19.183.254:45033)
0809 12:20:05.026125 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:05.026130 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:05.026643 (+ 513us) server_negotiation.cc:407] Connection header received
0809 12:20:05.026960 (+ 317us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:05.026965 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:05.027087 (+ 122us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:05.027239 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:20:05.027681 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.028339 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.029854 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.030298 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.030438 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:05.030996 (+ 558us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:20:05.031433 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:20:05.031606 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:05.031685 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 12:20:05.031762 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 12:20:05.032898 8876 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:48248:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:20:05.040031 8876 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:20:05.057214 9190 heartbeater.cc:503] Master 127.19.183.254:45033 was elected leader, sending a full tablet report...
I20260809 12:20:05.061895 9587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:05.051328 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43569 (local address 127.0.0.1:50642)
0809 12:20:05.052985 (+ 1657us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:05.053033 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:05.053243 (+ 210us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:05.054345 (+ 1102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:05.054351 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:05.054374 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:20:05.054751 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:05.054764 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.056861 (+ 2097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.056866 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:05.059606 (+ 2740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:05.059620 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.060887 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.060893 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:05.061101 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:05.061161 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 12:20:05.061323 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 12:20:05.061511 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1439,"thread_start_us":1302,"threads_started":1}
I20260809 12:20:05.061877 9589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:05.051787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50642 (local address 127.19.183.193:43569)
0809 12:20:05.052557 (+ 770us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:05.052566 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:05.053167 (+ 601us) server_negotiation.cc:407] Connection header received
0809 12:20:05.053475 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:05.053485 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:05.053749 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:05.054089 (+ 340us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:20:05.054976 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.056714 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.059757 (+ 3043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.060665 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.060882 (+ 217us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:05.061109 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:05.061369 (+ 260us) server_negotiation.cc:299] Negotiation successful
0809 12:20:05.061536 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":566,"thread_start_us":401,"threads_started":1}
I20260809 12:20:05.070365 9587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:05.054818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:37603 (local address 127.0.0.1:47422)
0809 12:20:05.062125 (+ 7307us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:05.062143 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:05.062258 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:05.063147 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:05.063152 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:05.063178 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:20:05.063374 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:05.063385 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.065145 (+ 1760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.065152 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:05.067041 (+ 1889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:05.067056 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.069416 (+ 2360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.069422 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:05.069685 (+ 263us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:05.069725 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 12:20:05.069830 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 12:20:05.069990 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7209,"mutex_wait_us":44,"thread_start_us":10340,"threads_started":1}
I20260809 12:20:05.070415 9593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:05.054442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47422 (local address 127.19.183.195:37603)
0809 12:20:05.055497 (+ 1055us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:05.055508 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:05.062245 (+ 6737us) server_negotiation.cc:407] Connection header received
0809 12:20:05.062644 (+ 399us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:05.062655 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:05.062957 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:05.063166 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:20:05.063581 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.064957 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.067219 (+ 2262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.068908 (+ 1689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.069111 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:05.069287 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:05.069902 (+ 615us) server_negotiation.cc:299] Negotiation successful
0809 12:20:05.070073 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":880,"thread_start_us":726,"threads_started":1}
I20260809 12:20:05.069954 9080 tablet_service.cc:1467] Processing CreateTablet for tablet e8071ba36e0c4bbd9268c3e8e2b3039e (DEFAULT_TABLE table=test-table [id=6ee754701727424882101511a16933c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:20:05.071956 9080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8071ba36e0c4bbd9268c3e8e2b3039e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:20:05.073307 9590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:05.053433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:38419 (local address 127.0.0.1:45062)
0809 12:20:05.054029 (+ 596us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:05.054046 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:05.054147 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:05.066747 (+ 12600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:05.066752 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:05.066778 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:20:05.067205 (+ 427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:05.067218 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.068995 (+ 1777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.069000 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:05.070822 (+ 1822us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:05.070837 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.072458 (+ 1621us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.072465 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:05.072659 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:05.072695 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 12:20:05.072798 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 12:20:05.072925 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":39,"thread_start_us":317,"threads_started":1}
I20260809 12:20:05.073356 9592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:05.053762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45062 (local address 127.19.183.194:38419)
0809 12:20:05.066140 (+ 12378us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:05.066149 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:05.066186 (+ 37us) server_negotiation.cc:407] Connection header received
0809 12:20:05.066263 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:05.066268 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:05.066434 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:05.066742 (+ 308us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:20:05.067395 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.068794 (+ 1399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.071010 (+ 2216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.072241 (+ 1231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.072436 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:05.072608 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:05.072856 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 12:20:05.073026 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12210,"thread_start_us":12099,"threads_started":1}
I20260809 12:20:05.074259 9476 tablet_service.cc:1467] Processing CreateTablet for tablet e8071ba36e0c4bbd9268c3e8e2b3039e (DEFAULT_TABLE table=test-table [id=6ee754701727424882101511a16933c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:20:05.075773 9476 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8071ba36e0c4bbd9268c3e8e2b3039e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:20:05.077319 9289 tablet_service.cc:1467] Processing CreateTablet for tablet e8071ba36e0c4bbd9268c3e8e2b3039e (DEFAULT_TABLE table=test-table [id=6ee754701727424882101511a16933c2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:20:05.078668 9289 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e8071ba36e0c4bbd9268c3e8e2b3039e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:20:05.088996 9597 tablet_bootstrap.cc:492] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865: Bootstrap starting.
I20260809 12:20:05.091472 9598 tablet_bootstrap.cc:492] T e8071ba36e0c4bbd9268c3e8e2b3039e P 41979697f97c4989bbf976682219d2c7: Bootstrap starting.
I20260809 12:20:05.093659 9600 tablet_bootstrap.cc:492] T e8071ba36e0c4bbd9268c3e8e2b3039e P 819e376295694741baea9fe748936978: Bootstrap starting.
I20260809 12:20:05.093837 9597 tablet_bootstrap.cc:654] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865: Neither blocks nor log segments found. Creating new log.
I20260809 12:20:05.095173 9598 tablet_bootstrap.cc:654] T e8071ba36e0c4bbd9268c3e8e2b3039e P 41979697f97c4989bbf976682219d2c7: Neither blocks nor log segments found. Creating new log.
I20260809 12:20:05.095230 9597 log.cc:824] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865: Log is configured to *not* fsync() on all Append() calls
I20260809 12:20:05.096352 9598 log.cc:824] T e8071ba36e0c4bbd9268c3e8e2b3039e P 41979697f97c4989bbf976682219d2c7: Log is configured to *not* fsync() on all Append() calls
I20260809 12:20:05.096818 9600 tablet_bootstrap.cc:654] T e8071ba36e0c4bbd9268c3e8e2b3039e P 819e376295694741baea9fe748936978: Neither blocks nor log segments found. Creating new log.
I20260809 12:20:05.098057 9600 log.cc:824] T e8071ba36e0c4bbd9268c3e8e2b3039e P 819e376295694741baea9fe748936978: Log is configured to *not* fsync() on all Append() calls
I20260809 12:20:05.098681 9597 tablet_bootstrap.cc:492] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865: No bootstrap required, opened a new log
I20260809 12:20:05.099006 9597 ts_tablet_manager.cc:1397] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.002s
I20260809 12:20:05.099582 9598 tablet_bootstrap.cc:492] T e8071ba36e0c4bbd9268c3e8e2b3039e P 41979697f97c4989bbf976682219d2c7: No bootstrap required, opened a new log
I20260809 12:20:05.099840 9598 ts_tablet_manager.cc:1397] T e8071ba36e0c4bbd9268c3e8e2b3039e P 41979697f97c4989bbf976682219d2c7: Time spent bootstrapping tablet: real 0.009s user 0.004s sys 0.003s
I20260809 12:20:05.100762 9600 tablet_bootstrap.cc:492] T e8071ba36e0c4bbd9268c3e8e2b3039e P 819e376295694741baea9fe748936978: No bootstrap required, opened a new log
I20260809 12:20:05.101039 9600 ts_tablet_manager.cc:1397] T e8071ba36e0c4bbd9268c3e8e2b3039e P 819e376295694741baea9fe748936978: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260809 12:20:05.108292 9600 raft_consensus.cc:348] T e8071ba36e0c4bbd9268c3e8e2b3039e P 819e376295694741baea9fe748936978 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f3e4b638b13408a95ef66cdb91b2865" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43569 } } peers { permanent_uuid: "819e376295694741baea9fe748936978" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38419 } } peers { permanent_uuid: "41979697f97c4989bbf976682219d2c7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 37603 } }
I20260809 12:20:05.108693 9600 raft_consensus.cc:374] T e8071ba36e0c4bbd9268c3e8e2b3039e P 819e376295694741baea9fe748936978 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:20:05.108800 9600 raft_consensus.cc:729] T e8071ba36e0c4bbd9268c3e8e2b3039e P 819e376295694741baea9fe748936978 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 819e376295694741baea9fe748936978, State: Initialized, Role: FOLLOWER
I20260809 12:20:05.109442 9600 consensus_queue.cc:260] T e8071ba36e0c4bbd9268c3e8e2b3039e P 819e376295694741baea9fe748936978 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f3e4b638b13408a95ef66cdb91b2865" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43569 } } peers { permanent_uuid: "819e376295694741baea9fe748936978" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38419 } } peers { permanent_uuid: "41979697f97c4989bbf976682219d2c7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 37603 } }
I20260809 12:20:05.109699 9597 raft_consensus.cc:348] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f3e4b638b13408a95ef66cdb91b2865" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43569 } } peers { permanent_uuid: "819e376295694741baea9fe748936978" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38419 } } peers { permanent_uuid: "41979697f97c4989bbf976682219d2c7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 37603 } }
I20260809 12:20:05.110145 9597 raft_consensus.cc:374] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:20:05.110266 9597 raft_consensus.cc:729] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f3e4b638b13408a95ef66cdb91b2865, State: Initialized, Role: FOLLOWER
I20260809 12:20:05.110797 9386 heartbeater.cc:503] Master 127.19.183.254:45033 was elected leader, sending a full tablet report...
I20260809 12:20:05.110977 9597 consensus_queue.cc:260] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f3e4b638b13408a95ef66cdb91b2865" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43569 } } peers { permanent_uuid: "819e376295694741baea9fe748936978" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38419 } } peers { permanent_uuid: "41979697f97c4989bbf976682219d2c7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 37603 } }
I20260809 12:20:05.111011 9598 raft_consensus.cc:348] T e8071ba36e0c4bbd9268c3e8e2b3039e P 41979697f97c4989bbf976682219d2c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f3e4b638b13408a95ef66cdb91b2865" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43569 } } peers { permanent_uuid: "819e376295694741baea9fe748936978" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38419 } } peers { permanent_uuid: "41979697f97c4989bbf976682219d2c7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 37603 } }
I20260809 12:20:05.111414 9598 raft_consensus.cc:374] T e8071ba36e0c4bbd9268c3e8e2b3039e P 41979697f97c4989bbf976682219d2c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:20:05.111454 9600 ts_tablet_manager.cc:1428] T e8071ba36e0c4bbd9268c3e8e2b3039e P 819e376295694741baea9fe748936978: Time spent starting tablet: real 0.010s user 0.007s sys 0.004s
I20260809 12:20:05.111541 9598 raft_consensus.cc:729] T e8071ba36e0c4bbd9268c3e8e2b3039e P 41979697f97c4989bbf976682219d2c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 41979697f97c4989bbf976682219d2c7, State: Initialized, Role: FOLLOWER
I20260809 12:20:05.112247 9598 consensus_queue.cc:260] T e8071ba36e0c4bbd9268c3e8e2b3039e P 41979697f97c4989bbf976682219d2c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f3e4b638b13408a95ef66cdb91b2865" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43569 } } peers { permanent_uuid: "819e376295694741baea9fe748936978" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38419 } } peers { permanent_uuid: "41979697f97c4989bbf976682219d2c7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 37603 } }
I20260809 12:20:05.114919 9571 heartbeater.cc:503] Master 127.19.183.254:45033 was elected leader, sending a full tablet report...
I20260809 12:20:05.115816 9598 ts_tablet_manager.cc:1428] T e8071ba36e0c4bbd9268c3e8e2b3039e P 41979697f97c4989bbf976682219d2c7: Time spent starting tablet: real 0.016s user 0.015s sys 0.001s
I20260809 12:20:05.119067 9597 ts_tablet_manager.cc:1428] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865: Time spent starting tablet: real 0.020s user 0.018s sys 0.002s
W20260809 12:20:05.237005 9572 tablet.cc:2369] T e8071ba36e0c4bbd9268c3e8e2b3039e P 41979697f97c4989bbf976682219d2c7: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:20:05.245306 9387 tablet.cc:2369] T e8071ba36e0c4bbd9268c3e8e2b3039e P 819e376295694741baea9fe748936978: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:20:05.271911 9191 tablet.cc:2369] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:20:05.321743 9606 raft_consensus.cc:482] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:20:05.322209 9606 raft_consensus.cc:504] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f3e4b638b13408a95ef66cdb91b2865" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43569 } } peers { permanent_uuid: "819e376295694741baea9fe748936978" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38419 } } peers { permanent_uuid: "41979697f97c4989bbf976682219d2c7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 37603 } }
I20260809 12:20:05.324059 9606 leader_election.cc:288] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 819e376295694741baea9fe748936978 (127.19.183.194:38419), 41979697f97c4989bbf976682219d2c7 (127.19.183.195:37603)
I20260809 12:20:05.331853 9592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:05.324680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:54463 (local address 127.19.183.194:38419)
0809 12:20:05.324929 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:05.324939 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:05.325510 (+ 571us) server_negotiation.cc:407] Connection header received
0809 12:20:05.325818 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:05.325827 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:05.325912 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:05.325987 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:20:05.326758 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.327862 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.329841 (+ 1979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.331026 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.331149 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:05.331202 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:05.331570 (+ 368us) server_negotiation.cc:299] Negotiation successful
0809 12:20:05.331664 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:20:05.332087 9608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:05.324670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:38419 (local address 127.19.183.193:54463)
0809 12:20:05.325346 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:05.325389 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:05.325564 (+ 175us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:05.326168 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:05.326172 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:05.326194 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:20:05.326568 (+ 374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:05.326578 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.327985 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.327990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:05.329679 (+ 1689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:05.329693 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.331222 (+ 1529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.331228 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:05.331401 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:05.331437 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 12:20:05.331542 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 12:20:05.331676 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":403,"threads_started":1}
I20260809 12:20:05.332860 9593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:05.325645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:58213 (local address 127.19.183.195:37603)
0809 12:20:05.325872 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:05.325878 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:05.326805 (+ 927us) server_negotiation.cc:407] Connection header received
0809 12:20:05.327009 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:05.327013 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:05.327070 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:05.327141 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:20:05.327724 (+ 583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.328751 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.330745 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.331843 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.332011 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:05.332119 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:05.332488 (+ 369us) server_negotiation.cc:299] Negotiation successful
0809 12:20:05.332630 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 12:20:05.332895 9610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:05.325439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:37603 (local address 127.19.183.193:58213)
0809 12:20:05.326672 (+ 1233us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:05.326692 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:05.326858 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:05.327190 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:05.327195 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:05.327221 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:20:05.327558 (+ 337us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:05.327568 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.328897 (+ 1329us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.328901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:05.330536 (+ 1635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:05.330552 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.332115 (+ 1563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.332124 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:05.332300 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:05.332333 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 12:20:05.332427 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 12:20:05.332554 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1096,"thread_start_us":840,"threads_started":1}
I20260809 12:20:05.333333 9320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8071ba36e0c4bbd9268c3e8e2b3039e" candidate_uuid: "4f3e4b638b13408a95ef66cdb91b2865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "819e376295694741baea9fe748936978" is_pre_election: true
I20260809 12:20:05.333745 9499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8071ba36e0c4bbd9268c3e8e2b3039e" candidate_uuid: "4f3e4b638b13408a95ef66cdb91b2865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41979697f97c4989bbf976682219d2c7" is_pre_election: true
I20260809 12:20:05.333847 9320 raft_consensus.cc:2454] T e8071ba36e0c4bbd9268c3e8e2b3039e P 819e376295694741baea9fe748936978 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f3e4b638b13408a95ef66cdb91b2865 in term 0.
I20260809 12:20:05.334233 9499 raft_consensus.cc:2454] T e8071ba36e0c4bbd9268c3e8e2b3039e P 41979697f97c4989bbf976682219d2c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f3e4b638b13408a95ef66cdb91b2865 in term 0.
I20260809 12:20:05.334563 8998 leader_election.cc:302] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f3e4b638b13408a95ef66cdb91b2865, 819e376295694741baea9fe748936978; no voters:
I20260809 12:20:05.334961 9606 raft_consensus.cc:2781] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:20:05.335116 9606 raft_consensus.cc:482] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:20:05.335281 9606 raft_consensus.cc:3037] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:20:05.337504 9606 raft_consensus.cc:504] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f3e4b638b13408a95ef66cdb91b2865" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43569 } } peers { permanent_uuid: "819e376295694741baea9fe748936978" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38419 } } peers { permanent_uuid: "41979697f97c4989bbf976682219d2c7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 37603 } }
I20260809 12:20:05.338083 9606 leader_election.cc:288] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865 [CANDIDATE]: Term 1 election: Requested vote from peers 819e376295694741baea9fe748936978 (127.19.183.194:38419), 41979697f97c4989bbf976682219d2c7 (127.19.183.195:37603)
I20260809 12:20:05.338557 9320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8071ba36e0c4bbd9268c3e8e2b3039e" candidate_uuid: "4f3e4b638b13408a95ef66cdb91b2865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "819e376295694741baea9fe748936978"
I20260809 12:20:05.338649 9499 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e8071ba36e0c4bbd9268c3e8e2b3039e" candidate_uuid: "4f3e4b638b13408a95ef66cdb91b2865" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "41979697f97c4989bbf976682219d2c7"
I20260809 12:20:05.338781 9320 raft_consensus.cc:3037] T e8071ba36e0c4bbd9268c3e8e2b3039e P 819e376295694741baea9fe748936978 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:20:05.338918 9499 raft_consensus.cc:3037] T e8071ba36e0c4bbd9268c3e8e2b3039e P 41979697f97c4989bbf976682219d2c7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:20:05.341084 9499 raft_consensus.cc:2454] T e8071ba36e0c4bbd9268c3e8e2b3039e P 41979697f97c4989bbf976682219d2c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f3e4b638b13408a95ef66cdb91b2865 in term 1.
I20260809 12:20:05.341087 9320 raft_consensus.cc:2454] T e8071ba36e0c4bbd9268c3e8e2b3039e P 819e376295694741baea9fe748936978 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f3e4b638b13408a95ef66cdb91b2865 in term 1.
I20260809 12:20:05.341482 8998 leader_election.cc:302] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4f3e4b638b13408a95ef66cdb91b2865, 819e376295694741baea9fe748936978; no voters:
I20260809 12:20:05.341804 9606 raft_consensus.cc:2781] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:20:05.342240 9606 raft_consensus.cc:686] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865 [term 1 LEADER]: Becoming Leader. State: Replica: 4f3e4b638b13408a95ef66cdb91b2865, State: Running, Role: LEADER
I20260809 12:20:05.342820 9606 consensus_queue.cc:237] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f3e4b638b13408a95ef66cdb91b2865" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43569 } } peers { permanent_uuid: "819e376295694741baea9fe748936978" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38419 } } peers { permanent_uuid: "41979697f97c4989bbf976682219d2c7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 37603 } }
I20260809 12:20:05.347957 8876 catalog_manager.cc:5495] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f3e4b638b13408a95ef66cdb91b2865 (127.19.183.193). New cstate: current_term: 1 leader_uuid: "4f3e4b638b13408a95ef66cdb91b2865" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4f3e4b638b13408a95ef66cdb91b2865" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43569 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "819e376295694741baea9fe748936978" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "41979697f97c4989bbf976682219d2c7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 37603 } health_report { overall_health: UNKNOWN } } }
I20260809 12:20:05.428248 9589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:05.422799 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50656 (local address 127.19.183.193:43569)
0809 12:20:05.423022 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:05.423027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:05.423047 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:20:05.423221 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:05.423225 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:05.423304 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:05.423470 (+ 166us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:20:05.423862 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.424556 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.426005 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.426412 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.426555 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:05.427135 (+ 580us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:20:05.427624 (+ 489us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:20:05.427792 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:05.427871 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 12:20:05.427951 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 12:20:05.436326 9499 raft_consensus.cc:1261] T e8071ba36e0c4bbd9268c3e8e2b3039e P 41979697f97c4989bbf976682219d2c7 [term 1 FOLLOWER]: Refusing update from remote peer 4f3e4b638b13408a95ef66cdb91b2865: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:20:05.436369 9320 raft_consensus.cc:1261] T e8071ba36e0c4bbd9268c3e8e2b3039e P 819e376295694741baea9fe748936978 [term 1 FOLLOWER]: Refusing update from remote peer 4f3e4b638b13408a95ef66cdb91b2865: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:20:05.437332 9606 consensus_queue.cc:1035] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "41979697f97c4989bbf976682219d2c7" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 37603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:20:05.437642 9613 consensus_queue.cc:1035] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865 [LEADER]: Connected to new peer: Peer: permanent_uuid: "819e376295694741baea9fe748936978" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 38419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:20:05.451224 9619 mvcc.cc:205] Tried to move back new op lower bound from 7316594710256955392 to 7316594709888700416. Current Snapshot: MvccSnapshot[applied={T|T < 7316594710256955392}]
I20260809 12:20:05.462837 8879 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:48248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:20:05.463088 8879 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:48248:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:20:05.466799 8879 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0: Sending DeleteTablet for 3 replicas of tablet e8071ba36e0c4bbd9268c3e8e2b3039e
I20260809 12:20:05.467967 9080 tablet_service.cc:1514] Processing DeleteTablet for tablet e8071ba36e0c4bbd9268c3e8e2b3039e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:20:05 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:50642
I20260809 12:20:05.468122 9289 tablet_service.cc:1514] Processing DeleteTablet for tablet e8071ba36e0c4bbd9268c3e8e2b3039e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:20:05 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:45062
I20260809 12:20:05.468230 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 8941
I20260809 12:20:05.469065 9476 tablet_service.cc:1514] Processing DeleteTablet for tablet e8071ba36e0c4bbd9268c3e8e2b3039e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:20:05 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:47422
I20260809 12:20:05.478027 9631 tablet_replica.cc:330] T e8071ba36e0c4bbd9268c3e8e2b3039e P 819e376295694741baea9fe748936978: stopping tablet replica
I20260809 12:20:05.478252 9630 tablet_replica.cc:330] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865: stopping tablet replica
I20260809 12:20:05.478507 9631 raft_consensus.cc:2229] T e8071ba36e0c4bbd9268c3e8e2b3039e P 819e376295694741baea9fe748936978 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:20:05.478731 9630 raft_consensus.cc:2229] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:20:05.479097 9631 raft_consensus.cc:2258] T e8071ba36e0c4bbd9268c3e8e2b3039e P 819e376295694741baea9fe748936978 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:20:05.479502 9630 raft_consensus.cc:2258] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:20:05.481621 9630 ts_tablet_manager.cc:1905] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:20:05.481940 9631 ts_tablet_manager.cc:1905] T e8071ba36e0c4bbd9268c3e8e2b3039e P 819e376295694741baea9fe748936978: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:20:05.484401 9632 tablet_replica.cc:330] T e8071ba36e0c4bbd9268c3e8e2b3039e P 41979697f97c4989bbf976682219d2c7: stopping tablet replica
I20260809 12:20:05.484867 9632 raft_consensus.cc:2229] T e8071ba36e0c4bbd9268c3e8e2b3039e P 41979697f97c4989bbf976682219d2c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:20:05.485426 9632 raft_consensus.cc:2258] T e8071ba36e0c4bbd9268c3e8e2b3039e P 41979697f97c4989bbf976682219d2c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:20:05.487526 9632 ts_tablet_manager.cc:1905] T e8071ba36e0c4bbd9268c3e8e2b3039e P 41979697f97c4989bbf976682219d2c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:20:05.488996 9631 ts_tablet_manager.cc:1918] T e8071ba36e0c4bbd9268c3e8e2b3039e P 819e376295694741baea9fe748936978: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:20:05.489164 9631 log.cc:1196] T e8071ba36e0c4bbd9268c3e8e2b3039e P 819e376295694741baea9fe748936978: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/e8071ba36e0c4bbd9268c3e8e2b3039e
I20260809 12:20:05.489809 9631 ts_tablet_manager.cc:1939] T e8071ba36e0c4bbd9268c3e8e2b3039e P 819e376295694741baea9fe748936978: Deleting consensus metadata
I20260809 12:20:05.490038 9630 ts_tablet_manager.cc:1918] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:20:05.490307 9630 log.cc:1196] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/e8071ba36e0c4bbd9268c3e8e2b3039e
I20260809 12:20:05.491849 8853 catalog_manager.cc:4841] TS 819e376295694741baea9fe748936978 (127.19.183.194:38419): tablet e8071ba36e0c4bbd9268c3e8e2b3039e (table test-table [id=6ee754701727424882101511a16933c2]) successfully deleted
I20260809 12:20:05.493319 9589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:05.469118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50670 (local address 127.19.183.193:43569)
0809 12:20:05.469331 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:05.469340 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:05.469368 (+ 28us) server_negotiation.cc:407] Connection header received
0809 12:20:05.469669 (+ 301us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:05.469677 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:05.469780 (+ 103us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:05.469914 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:05.472208 (+ 2294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.473946 (+ 1738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.476640 (+ 2694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.477533 (+ 893us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.477717 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:05.483535 (+ 5818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:05.483554 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:05.483581 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:05.483651 (+ 70us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:05.488057 (+ 4406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:05.489192 (+ 1135us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:05.489200 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:05.489203 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:05.489368 (+ 165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:05.491157 (+ 1789us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:05.491162 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:05.491167 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:05.491724 (+ 557us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:05.492051 (+ 327us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:05.492684 (+ 633us) server_negotiation.cc:299] Negotiation successful
0809 12:20:05.492934 (+ 250us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 12:20:05.494855 9632 ts_tablet_manager.cc:1918] T e8071ba36e0c4bbd9268c3e8e2b3039e P 41979697f97c4989bbf976682219d2c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:20:05.495018 9632 log.cc:1196] T e8071ba36e0c4bbd9268c3e8e2b3039e P 41979697f97c4989bbf976682219d2c7: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/e8071ba36e0c4bbd9268c3e8e2b3039e
I20260809 12:20:05.495590 9632 ts_tablet_manager.cc:1939] T e8071ba36e0c4bbd9268c3e8e2b3039e P 41979697f97c4989bbf976682219d2c7: Deleting consensus metadata
I20260809 12:20:05.495584 9184 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:20:05.497740 8853 catalog_manager.cc:4841] TS 41979697f97c4989bbf976682219d2c7 (127.19.183.195:37603): tablet e8071ba36e0c4bbd9268c3e8e2b3039e (table test-table [id=6ee754701727424882101511a16933c2]) successfully deleted
I20260809 12:20:05.593763 9630 ts_tablet_manager.cc:1939] T e8071ba36e0c4bbd9268c3e8e2b3039e P 4f3e4b638b13408a95ef66cdb91b2865: Deleting consensus metadata
I20260809 12:20:05.595119 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 8941
I20260809 12:20:05.625093 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 9197
W20260809 12:20:05.625149 8853 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.183.193:43569 (error 108)
I20260809 12:20:05.627331 9639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:05.626462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43569 (local address 127.0.0.1:50680)
0809 12:20:05.626959 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:05.627087 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":247,"threads_started":1}
W20260809 12:20:05.627631 8853 catalog_manager.cc:4584] TS 4f3e4b638b13408a95ef66cdb91b2865 (127.19.183.193:43569): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e8071ba36e0c4bbd9268c3e8e2b3039e: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43569: connect: Connection refused (error 111)
I20260809 12:20:05.638934 9592 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:05.626035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45070 (local address 127.19.183.194:38419)
0809 12:20:05.626261 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:05.626279 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:05.626334 (+ 55us) server_negotiation.cc:407] Connection header received
0809 12:20:05.626567 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:05.626575 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:05.626683 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:05.626825 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:05.628443 (+ 1618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.629144 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.630307 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.630688 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.631079 (+ 391us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:05.633610 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:05.633624 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:05.633638 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:05.633673 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:05.636427 (+ 2754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:05.637046 (+ 619us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:05.637055 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:05.637057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:05.637192 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:05.637510 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:05.637521 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:05.637523 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:05.637893 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:05.638100 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:05.638387 (+ 287us) server_negotiation.cc:299] Negotiation successful
0809 12:20:05.638555 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:20:05.639720 9381 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:20:05.681212 9639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:05.680744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43569 (local address 127.0.0.1:50688)
0809 12:20:05.680964 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:05.681052 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260809 12:20:05.682281 9639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:05.681918 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43569 (local address 127.0.0.1:50700)
0809 12:20:05.682084 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:05.682152 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 12:20:05.730111 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 9197
I20260809 12:20:05.742808 9639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:05.742290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43569 (local address 127.0.0.1:50708)
0809 12:20:05.742511 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:05.742608 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 12:20:05.743855 9639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:05.743516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43569 (local address 127.0.0.1:50712)
0809 12:20:05.743663 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:05.743733 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260809 12:20:05.758257 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 9389
I20260809 12:20:05.771616 9593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:05.759150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47434 (local address 127.19.183.195:37603)
0809 12:20:05.759329 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:05.759334 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:05.759353 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:20:05.759491 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:05.759494 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:05.759563 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:05.759654 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:05.761015 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.761717 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.763426 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:05.763791 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:05.763886 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:05.766366 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:05.766381 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:05.766399 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:05.766437 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:05.769127 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:05.769784 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:05.769791 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:05.769793 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:05.769939 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:05.770358 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:05.770362 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:05.770364 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:05.770752 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:05.770991 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:05.771192 (+ 201us) server_negotiation.cc:299] Negotiation successful
0809 12:20:05.771343 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 12:20:05.772272 9564 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:20:05.824474 9639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:05.823907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43569 (local address 127.0.0.1:50722)
0809 12:20:05.824174 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:05.824283 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":92}
I20260809 12:20:05.825754 9639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:05.825285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43569 (local address 127.0.0.1:50730)
0809 12:20:05.825501 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:05.825585 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109}
I20260809 12:20:05.861476 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 9389
I20260809 12:20:05.885720 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 8830
I20260809 12:20:05.887022 8928 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:20:06.000298 9639 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:05.999676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43569 (local address 127.0.0.1:50746)
0809 12:20:05.999906 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:06.000034 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43569: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 12:20:06.000310 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 8830
I20260809 12:20:06.034956 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:45033
--webserver_interface=127.19.183.254
--webserver_port=35351
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:46539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:45033
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:20:06.384698 9643 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:20:06.385098 9643 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:20:06.385275 9643 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:20:06.393267 9643 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:20:06.393420 9643 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:20:06.393505 9643 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:20:06.393591 9643 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:20:06.393657 9643 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:20:06.403510 9643 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:46539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:45033
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:45033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=35351
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:20:06.404973 9643 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:20:06.406381 9643 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:20:06.418265 9655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:06.418327 9651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:06.418779 9649 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:20:06.419555 9643 server_base.cc:1034] running on GCE node
I20260809 12:20:06.420285 9643 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:20:06.421687 9643 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:20:06.422930 9643 hybrid_clock.cc:648] HybridClock initialized: now 1786278006422874 us; error 63 us; skew 500 ppm
I20260809 12:20:06.423362 9643 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:20:06 dist-test-slave-39l8 krb5kdc[7994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278006, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:20:06.426415 9643 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:20:06.427838 9643 webserver.cc:460] Webserver started at http://127.19.183.254:35351/ using document root <none> and password file <none>
I20260809 12:20:06.428459 9643 fs_manager.cc:359] Metadata directory not provided
I20260809 12:20:06.428560 9643 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:20:06.433748 9643 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.002s
I20260809 12:20:06.436617 9670 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:06.437896 9643 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.001s
I20260809 12:20:06.438036 9643 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "c8a0592aa088438ea86a0bbb545d93c0"
format_stamp: "Formatted at 2026-08-09 12:20:00 on dist-test-slave-39l8"
server_key: "bdf58f8d2789001a231b04dfb4fc1d30"
server_key_iv: "04357a4fa0b65ba1a45d16c7cbaf5c2a"
server_key_version: "encryptionkey@0"
I20260809 12:20:06.439082 9643 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:20:06.463521 9643 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:20:06.467484 9643 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:20:06.467705 9643 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:20:06.486954 9643 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:45033
I20260809 12:20:06.486981 9739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:45033 every 8 connection(s)
I20260809 12:20:06.488547 9643 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:20:06.494372 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 9643
I20260809 12:20:06.495513 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:43569
--local_ip_for_outbound_sockets=127.19.183.193
--tserver_master_addrs=127.19.183.254:45033
--webserver_port=42227
--webserver_interface=127.19.183.193
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:46539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:20:06.495939 9741 sys_catalog.cc:263] Verifying existing consensus state
I20260809 12:20:06.499235 9741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0: Bootstrap starting.
I20260809 12:20:06.524048 9741 log.cc:824] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0: Log is configured to *not* fsync() on all Append() calls
I20260809 12:20:06.546958 9741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:20:06.547627 9741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0: Bootstrap complete.
I20260809 12:20:06.561293 9741 raft_consensus.cc:348] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0592aa088438ea86a0bbb545d93c0" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45033 } }
I20260809 12:20:06.562674 9741 raft_consensus.cc:729] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: c8a0592aa088438ea86a0bbb545d93c0, State: Initialized, Role: FOLLOWER
I20260809 12:20:06.563463 9741 consensus_queue.cc:260] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0592aa088438ea86a0bbb545d93c0" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45033 } }
I20260809 12:20:06.563776 9741 raft_consensus.cc:388] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:20:06.563917 9741 raft_consensus.cc:482] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:20:06.564109 9741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 2 FOLLOWER]: Advancing to term 3
I20260809 12:20:06.570084 9741 raft_consensus.cc:504] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0592aa088438ea86a0bbb545d93c0" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45033 } }
I20260809 12:20:06.570745 9741 leader_election.cc:302] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c8a0592aa088438ea86a0bbb545d93c0; no voters:
I20260809 12:20:06.571461 9741 leader_election.cc:288] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 12:20:06.571604 9745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 12:20:06.573122 9745 raft_consensus.cc:686] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [term 3 LEADER]: Becoming Leader. State: Replica: c8a0592aa088438ea86a0bbb545d93c0, State: Running, Role: LEADER
I20260809 12:20:06.573655 9745 consensus_queue.cc:237] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0592aa088438ea86a0bbb545d93c0" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45033 } }
I20260809 12:20:06.574036 9741 sys_catalog.cc:564] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:20:06.576327 9749 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c8a0592aa088438ea86a0bbb545d93c0. Latest consensus state: current_term: 3 leader_uuid: "c8a0592aa088438ea86a0bbb545d93c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0592aa088438ea86a0bbb545d93c0" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45033 } } }
I20260809 12:20:06.576750 9749 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [sys.catalog]: This master's current role is: LEADER
I20260809 12:20:06.577240 9748 sys_catalog.cc:455] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "c8a0592aa088438ea86a0bbb545d93c0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c8a0592aa088438ea86a0bbb545d93c0" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 45033 } } }
I20260809 12:20:06.577560 9748 sys_catalog.cc:458] T 00000000000000000000000000000000 P c8a0592aa088438ea86a0bbb545d93c0 [sys.catalog]: This master's current role is: LEADER
I20260809 12:20:06.581467 9756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:20:06.590899 9756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:20:06.592535 9756 catalog_manager.cc:1259] Loaded cluster ID: 54a46fa9988e4b1896acc48c1d2a09d1
I20260809 12:20:06.592710 9756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 12:20:06.595211 9756 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f1b59e899ad google::LogMessage::Fail() at ??:0
@ 0x7f1b59e8c88a google::LogMessage::SendToLog() at ??:0
@ 0x7f1b59e894de google::LogMessage::Flush() at ??:0
@ 0x7f1b59e8aa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f1b6f8bc512 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f1b6f8b9976 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f1b6f941610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f1b6f941439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f1b5cad5345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f1b5cad891c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f1b5cad8799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f1b5cab3eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f1b5fd8d6db start_thread at ??:0
@ 0x7f1b58d2361f clone at ??:0
W20260809 12:20:06.947592 9743 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:20:06.947889 9743 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:20:06.947996 9743 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:20:06.956037 9743 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:20:06.956161 9743 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:20:06.956286 9743 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:20:06.966885 9743 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:46539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:43569
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=42227
--tserver_master_addrs=127.19.183.254:45033
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:20:06.968143 9743 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:20:06.969452 9743 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:20:06.982968 9807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:06.982928 9812 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:06.983008 9808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:20:06.984059 9743 server_base.cc:1034] running on GCE node
I20260809 12:20:06.984584 9743 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:20:06.985471 9743 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:20:06.986665 9743 hybrid_clock.cc:648] HybridClock initialized: now 1786278006986611 us; error 46 us; skew 500 ppm
I20260809 12:20:06.986980 9743 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:20:06 dist-test-slave-39l8 krb5kdc[7994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278006, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:20:06.990072 9743 init.cc:377] Logged in from keytab as kudu/127.19.183.193@KRBTEST.COM (short username kudu)
I20260809 12:20:06.991657 9743 webserver.cc:460] Webserver started at http://127.19.183.193:42227/ using document root <none> and password file <none>
I20260809 12:20:06.992395 9743 fs_manager.cc:359] Metadata directory not provided
I20260809 12:20:06.992553 9743 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:20:06.997917 9743 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 12:20:07.000734 9821 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:07.002060 9743 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:20:07.002234 9743 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "4f3e4b638b13408a95ef66cdb91b2865"
format_stamp: "Formatted at 2026-08-09 12:20:01 on dist-test-slave-39l8"
server_key: "60ccc26f4fe3ebb0d68f6e558d5f911c"
server_key_iv: "277f1781e4341768f7e8f7490ac58b51"
server_key_version: "encryptionkey@0"
I20260809 12:20:07.003364 9743 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:20:07.019130 9743 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:20:07.022991 9743 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:20:07.023280 9743 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:20:07.024271 9743 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:20:07.025661 9743 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:20:07.025799 9743 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:07.025938 9743 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:20:07.026036 9743 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:07.061652 9743 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:43569
I20260809 12:20:07.061713 9982 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:43569 every 8 connection(s)
I20260809 12:20:07.063221 9743 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:20:07.063917 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 9743
I20260809 12:20:07.064770 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:38419
--local_ip_for_outbound_sockets=127.19.183.194
--tserver_master_addrs=127.19.183.254:45033
--webserver_port=36611
--webserver_interface=127.19.183.194
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:46539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:20:07.068507 9986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:07.066645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.193:36463)
0809 12:20:07.067468 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:07.067686 (+ 218us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":565,"thread_start_us":327,"threads_started":1}
W20260809 12:20:07.069236 9984 heartbeater.cc:644] Failed to heartbeat to 127.19.183.254:45033 (0 consecutive failures): Network error: Failed to ping master at 127.19.183.254:45033: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
I20260809 12:20:07.070161 9986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:07.069813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.193:55535)
0809 12:20:07.069964 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:07.070054 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 12:20:07.071055 9986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:07.070685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.193:53999)
0809 12:20:07.070855 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:07.070935 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
W20260809 12:20:07.071326 9984 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 12:20:07.420130 9988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:20:07.420404 9988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:20:07.420502 9988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:20:07.427714 9988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:20:07.427824 9988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:20:07.427939 9988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:20:07.437644 9988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:46539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:38419
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=36611
--tserver_master_addrs=127.19.183.254:45033
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:20:07.438928 9988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:20:07.440207 9988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:20:07.454789 9995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:07.454860 9999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:07.454967 9997 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:20:07.455994 9988 server_base.cc:1034] running on GCE node
I20260809 12:20:07.456686 9988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:20:07.457692 9988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:20:07.458921 9988 hybrid_clock.cc:648] HybridClock initialized: now 1786278007458882 us; error 67 us; skew 500 ppm
I20260809 12:20:07.459239 9988 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:20:07 dist-test-slave-39l8 krb5kdc[7994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278007, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:20:07.462458 9988 init.cc:377] Logged in from keytab as kudu/127.19.183.194@KRBTEST.COM (short username kudu)
I20260809 12:20:07.463994 9988 webserver.cc:460] Webserver started at http://127.19.183.194:36611/ using document root <none> and password file <none>
I20260809 12:20:07.464696 9988 fs_manager.cc:359] Metadata directory not provided
I20260809 12:20:07.464848 9988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:20:07.470194 9988 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:20:07.473052 10008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:07.474509 9988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 12:20:07.474690 9988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "819e376295694741baea9fe748936978"
format_stamp: "Formatted at 2026-08-09 12:20:01 on dist-test-slave-39l8"
server_key: "411bf0dd8e189818b91170f619d9139c"
server_key_iv: "cea7bb78442f32bdd02a4805ca12cb0b"
server_key_version: "encryptionkey@0"
I20260809 12:20:07.475771 9988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:20:07.494017 9988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:20:07.498456 9988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:20:07.498843 9988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:20:07.499888 9988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:20:07.501238 9988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:20:07.501379 9988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:07.501505 9988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:20:07.501590 9988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:07.536798 9988 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:38419
I20260809 12:20:07.536842 10169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:38419 every 8 connection(s)
I20260809 12:20:07.538367 9988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:20:07.543108 10172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:07.541284 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.194:50545)
0809 12:20:07.541999 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:07.542213 (+ 214us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":446,"thread_start_us":276,"threads_started":1}
W20260809 12:20:07.543757 10170 heartbeater.cc:644] Failed to heartbeat to 127.19.183.254:45033 (0 consecutive failures): Network error: Failed to ping master at 127.19.183.254:45033: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
I20260809 12:20:07.544546 10172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:07.544252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.194:40551)
0809 12:20:07.544381 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:07.544446 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 12:20:07.545375 10172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:07.545074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.194:49423)
0809 12:20:07.545184 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:07.545273 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 12:20:07.545661 10170 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 12:20:07.548180 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 9988
I20260809 12:20:07.548802 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:37603
--local_ip_for_outbound_sockets=127.19.183.195
--tserver_master_addrs=127.19.183.254:45033
--webserver_port=42115
--webserver_interface=127.19.183.195
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:46539
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:20:07.900442 10173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:20:07.900722 10173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:20:07.900818 10173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:20:07.908124 10173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:20:07.908308 10173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:20:07.908437 10173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:20:07.918386 10173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:46539
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:37603
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=42115
--tserver_master_addrs=127.19.183.254:45033
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:20:07.919790 10173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:20:07.921116 10173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:20:07.934844 10178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:07.934690 10182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:07.934759 10180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:20:07.935815 10173 server_base.cc:1034] running on GCE node
I20260809 12:20:07.936442 10173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:20:07.937397 10173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:20:07.938649 10173 hybrid_clock.cc:648] HybridClock initialized: now 1786278007938611 us; error 50 us; skew 500 ppm
I20260809 12:20:07.938997 10173 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:20:07 dist-test-slave-39l8 krb5kdc[7994](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278007, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:20:07.942076 10173 init.cc:377] Logged in from keytab as kudu/127.19.183.195@KRBTEST.COM (short username kudu)
I20260809 12:20:07.943648 10173 webserver.cc:460] Webserver started at http://127.19.183.195:42115/ using document root <none> and password file <none>
I20260809 12:20:07.944361 10173 fs_manager.cc:359] Metadata directory not provided
I20260809 12:20:07.944514 10173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:20:07.949944 10173 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 12:20:07.952704 10196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:07.954032 10173 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:20:07.954243 10173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "41979697f97c4989bbf976682219d2c7"
format_stamp: "Formatted at 2026-08-09 12:20:02 on dist-test-slave-39l8"
server_key: "1198e40b482b45556af63f4451ad2e34"
server_key_iv: "6ea27b718a6303f5b1af1927cc64e2fb"
server_key_version: "encryptionkey@0"
I20260809 12:20:07.955336 10173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:20:07.992348 10173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:20:07.996469 10173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:20:07.996766 10173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:20:07.997885 10173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:20:07.999295 10173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:20:07.999450 10173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:07.999576 10173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:20:07.999692 10173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:08.038823 10173 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:37603
I20260809 12:20:08.038889 10374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:37603 every 8 connection(s)
I20260809 12:20:08.040550 10173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:20:08.045441 10377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:08.043615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.195:57857)
0809 12:20:08.044351 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:08.044604 (+ 253us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":271,"threads_started":1}
W20260809 12:20:08.046177 10375 heartbeater.cc:644] Failed to heartbeat to 127.19.183.254:45033 (0 consecutive failures): Network error: Failed to ping master at 127.19.183.254:45033: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
I20260809 12:20:08.046976 10377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:08.046643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.195:36355)
0809 12:20:08.046818 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:08.046879 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 12:20:08.047839 10377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:08.047523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.195:60275)
0809 12:20:08.047641 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:08.047718 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260809 12:20:08.048096 10375 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 12:20:08.050987 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 10173
I20260809 12:20:08.072623 10380 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:08.071833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.193:49019)
0809 12:20:08.072274 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:08.072407 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":215,"threads_started":1}
I20260809 12:20:08.547231 10382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:08.546221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.194:59263)
0809 12:20:08.546849 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:08.546988 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":239,"threads_started":1}
I20260809 12:20:09.049556 10383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:09.048687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.195:49583)
0809 12:20:09.049177 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:09.049318 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":231,"threads_started":1}
I20260809 12:20:09.074554 10384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:09.073696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.193:38037)
0809 12:20:09.074282 (+ 586us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:09.074371 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":318,"threads_started":1}
I20260809 12:20:09.549191 10386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:09.548378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.194:50983)
0809 12:20:09.548902 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:09.548997 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":411,"thread_start_us":258,"threads_started":1}
I20260809 12:20:10.051245 10388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:10.050521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.195:38089)
0809 12:20:10.050988 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:10.051066 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":265,"threads_started":1}
I20260809 12:20:10.076181 10391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:10.075469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.193:47347)
0809 12:20:10.075926 (+ 457us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:10.076007 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":242,"threads_started":1}
I20260809 12:20:10.550858 10392 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:10.550088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.194:46637)
0809 12:20:10.550524 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:10.550605 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":224,"threads_started":1}
I20260809 12:20:11.052824 10393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:11.052144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.195:57247)
0809 12:20:11.052560 (+ 416us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:11.052652 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":210,"threads_started":1}
I20260809 12:20:11.078794 10395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:11.077069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.193:41925)
0809 12:20:11.078453 (+ 1384us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:11.078536 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1279,"thread_start_us":1156,"threads_started":1}
I20260809 12:20:11.552677 10396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:11.551748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.194:49655)
0809 12:20:11.552366 (+ 618us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:11.552451 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":480,"thread_start_us":295,"threads_started":1}
I20260809 12:20:12.054324 10397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:12.053665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.195:32917)
0809 12:20:12.054077 (+ 412us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:12.054157 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":216,"threads_started":1}
I20260809 12:20:12.080793 10398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:12.079872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.193:56211)
0809 12:20:12.080443 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:12.080551 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":283,"threads_started":1}
I20260809 12:20:12.554560 10399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:12.553769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.194:34889)
0809 12:20:12.554291 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:12.554369 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":287,"threads_started":1}
I20260809 12:20:13.056195 10400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:13.055410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.195:44049)
0809 12:20:13.055911 (+ 501us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:13.056013 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":236,"threads_started":1}
I20260809 12:20:13.082397 10402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:13.081629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.193:44927)
0809 12:20:13.082141 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:13.082232 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402,"thread_start_us":289,"threads_started":1}
I20260809 12:20:13.556248 10403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:13.555541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.194:43769)
0809 12:20:13.555976 (+ 435us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:13.556067 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":222,"threads_started":1}
I20260809 12:20:14.057754 10404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:14.057028 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.195:45373)
0809 12:20:14.057487 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:14.057577 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":236,"threads_started":1}
I20260809 12:20:14.084033 10405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:14.083235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.193:33331)
0809 12:20:14.083740 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:14.083818 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":216,"threads_started":1}
I20260809 12:20:14.558048 10406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:14.557171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.194:36079)
0809 12:20:14.557755 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:14.557835 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":294,"threads_started":1}
I20260809 12:20:15.059645 10407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:15.058819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.195:59139)
0809 12:20:15.059287 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:15.059364 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":194,"threads_started":1}
I20260809 12:20:15.085626 10408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:15.084954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.193:33277)
0809 12:20:15.085367 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:15.085450 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":217,"threads_started":1}
I20260809 12:20:15.560105 10409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:15.559045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.194:39063)
0809 12:20:15.559727 (+ 682us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:15.559863 (+ 136us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":552,"thread_start_us":385,"threads_started":1}
I20260809 12:20:16.061331 10411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:16.060615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.195:36095)
0809 12:20:16.061056 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:16.061160 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":236,"threads_started":1}
I20260809 12:20:16.087369 10412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:16.086548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.193:35085)
0809 12:20:16.087116 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:16.087194 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":456,"thread_start_us":322,"threads_started":1}
I20260809 12:20:16.562104 10414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:16.561126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.194:38353)
0809 12:20:16.561781 (+ 655us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:16.561904 (+ 123us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":275,"threads_started":1}
I20260809 12:20:17.063056 10415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:17.062161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.195:50601)
0809 12:20:17.062774 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:17.062871 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":299,"threads_started":1}
I20260809 12:20:17.089205 10416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:17.088428 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.193:55965)
0809 12:20:17.088934 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:17.089030 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":400,"thread_start_us":282,"threads_started":1}
I20260809 12:20:17.564000 10418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:17.563153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.194:59381)
0809 12:20:17.563725 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:17.563816 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":328,"threads_started":1}
I20260809 12:20:18.064878 10419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:18.064016 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.195:54319)
0809 12:20:18.064571 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:18.064705 (+ 134us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":308,"threads_started":1}
I20260809 12:20:18.091056 10421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:18.090201 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.193:54505)
0809 12:20:18.090769 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:18.090865 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":287,"threads_started":1}
I20260809 12:20:18.565845 10422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:18.565008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.194:51493)
0809 12:20:18.565547 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:18.565654 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":423,"thread_start_us":278,"threads_started":1}
I20260809 12:20:19.066826 10423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:19.065891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.195:38489)
0809 12:20:19.066468 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:19.066574 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":293,"threads_started":1}
I20260809 12:20:19.092921 10424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:19.092190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.193:43393)
0809 12:20:19.092664 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:19.092758 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":237,"threads_started":1}
I20260809 12:20:19.567899 10425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:19.567082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.194:43131)
0809 12:20:19.567637 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:19.567730 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":302,"threads_started":1}
I20260809 12:20:20.068755 10427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:20.067905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.195:49221)
0809 12:20:20.068488 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:20.068587 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":434,"thread_start_us":301,"threads_started":1}
I20260809 12:20:20.095033 10428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:20.094033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.193:56025)
0809 12:20:20.094660 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:20.094827 (+ 167us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":502,"thread_start_us":343,"threads_started":1}
I20260809 12:20:20.569885 10431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:20.568879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.194:60297)
0809 12:20:20.569564 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:20.569666 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":553,"thread_start_us":394,"threads_started":1}
I20260809 12:20:21.070928 10432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:21.069838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.195:42915)
0809 12:20:21.070531 (+ 693us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:21.070646 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":363,"threads_started":1}
I20260809 12:20:21.096954 10433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:21.096114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.193:60587)
0809 12:20:21.096669 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:21.096763 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":278,"threads_started":1}
I20260809 12:20:21.571930 10434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:21.571058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.194:38889)
0809 12:20:21.571626 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:21.571721 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":287,"threads_started":1}
I20260809 12:20:22.073004 10435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:22.072075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.195:38511)
0809 12:20:22.072707 (+ 632us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:22.072806 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":495,"thread_start_us":366,"threads_started":1}
I20260809 12:20:22.099200 10437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:22.098331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.193:34711)
0809 12:20:22.098882 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:22.098994 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":244,"threads_started":1}
I20260809 12:20:22.574195 10438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:22.573307 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.194:47151)
0809 12:20:22.573857 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:22.573972 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":303,"threads_started":1}
I20260809 12:20:23.058776 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 9743
I20260809 12:20:23.075253 10441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:23.074305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:45033 (local address 127.19.183.195:50903)
0809 12:20:23.074921 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:23.075030 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:45033: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":496,"thread_start_us":319,"threads_started":1}
I20260809 12:20:23.087158 10440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:23.060430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47792 (local address 127.19.183.193:43569)
0809 12:20:23.061170 (+ 740us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:23.061193 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:23.061273 (+ 80us) server_negotiation.cc:407] Connection header received
0809 12:20:23.064196 (+ 2923us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:23.064210 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:23.064841 (+ 631us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:23.065276 (+ 435us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:23.066989 (+ 1713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:23.068386 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:23.069815 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:23.070537 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:23.070761 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:23.079649 (+ 8888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:23.079661 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:23.079688 (+ 27us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:23.079743 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:23.083359 (+ 3616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:23.084258 (+ 899us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:23.084267 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:23.084274 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:23.084480 (+ 206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:23.084961 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:23.084966 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:23.084968 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:23.085609 (+ 641us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:23.085926 (+ 317us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:23.086347 (+ 421us) server_negotiation.cc:299] Negotiation successful
0809 12:20:23.086852 (+ 505us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":497,"thread_start_us":292,"threads_started":1}
I20260809 12:20:23.088788 9978 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:20:23.179054 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 9743
I20260809 12:20:23.210959 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 9988
I20260809 12:20:23.231247 10446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:23.211951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49590 (local address 127.19.183.194:38419)
0809 12:20:23.212640 (+ 689us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:23.212656 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:23.212708 (+ 52us) server_negotiation.cc:407] Connection header received
0809 12:20:23.214554 (+ 1846us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:23.214564 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:23.215136 (+ 572us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:23.215459 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:23.217042 (+ 1583us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:23.218332 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:23.219957 (+ 1625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:23.220729 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:23.220894 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:23.223929 (+ 3035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:23.223938 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:23.223956 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:23.224007 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:23.227469 (+ 3462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:23.228306 (+ 837us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:23.228315 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:23.228322 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:23.228519 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:23.228960 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:23.228964 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:23.228966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:23.229611 (+ 645us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:23.229893 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:23.230430 (+ 537us) server_negotiation.cc:299] Negotiation successful
0809 12:20:23.230929 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":457,"thread_start_us":286,"threads_started":1}
I20260809 12:20:23.232869 10165 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:20:23.322474 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 9988
I20260809 12:20:23.354195 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 10173
I20260809 12:20:23.373466 10452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:23.355377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48746 (local address 127.19.183.195:37603)
0809 12:20:23.356043 (+ 666us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:23.356058 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:23.356106 (+ 48us) server_negotiation.cc:407] Connection header received
0809 12:20:23.357836 (+ 1730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:23.357845 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:23.358295 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:23.358605 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:23.360193 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:23.361440 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:23.362822 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:23.363533 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:23.363764 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:23.366565 (+ 2801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:23.366573 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:23.366590 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:23.366645 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:23.369742 (+ 3097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:23.370534 (+ 792us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:23.370542 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:23.370549 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:23.370880 (+ 331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:23.371284 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:23.371289 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:23.371292 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:23.371876 (+ 584us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:23.372253 (+ 377us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:23.372673 (+ 420us) server_negotiation.cc:299] Negotiation successful
0809 12:20:23.373146 (+ 473us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":485,"thread_start_us":328,"threads_started":1}
I20260809 12:20:23.374981 10369 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:20:23.467749 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 10173
2026-08-09T12:20:23Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (24993 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:20:23 dist-test-slave-39l8 krb5kdc[10458](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:20:23 dist-test-slave-39l8 krb5kdc[10458](info): set up 2 sockets
Aug 09 12:20:23 dist-test-slave-39l8 krb5kdc[10458](info): commencing operation
krb5kdc: starting...
W20260809 12:20:24.808369 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.286s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:20:25 dist-test-slave-39l8 krb5kdc[10458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:20:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:20:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:20:25.140208 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:39553
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:39553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:20:25.501688 10483 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:20:25.501972 10483 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:20:25.502113 10483 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:20:25.509697 10483 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 12:20:25.509815 10483 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 12:20:25.509881 10483 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 12:20:25.509958 10483 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:20:25.510051 10483 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:20:25.510164 10483 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:20:25.510231 10483 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:20:25.510309 10483 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:20:25.520780 10483 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:39553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:39553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:20:25.522059 10483 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:20:25.523383 10483 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:20:25.535604 10490 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:25.535705 10491 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:25.535724 10497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:20:25.536010 10483 server_base.cc:1034] running on GCE node
I20260809 12:20:25.537127 10483 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:20:25.538522 10483 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:20:25.539736 10483 hybrid_clock.cc:648] HybridClock initialized: now 1786278025539659 us; error 50 us; skew 500 ppm
I20260809 12:20:25.540028 10483 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:20:25 dist-test-slave-39l8 krb5kdc[10458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278025, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:20:25.543113 10483 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:20:25.544520 10483 webserver.cc:460] Webserver started at http://127.19.183.254:43151/ using document root <none> and password file <none>
I20260809 12:20:25.545212 10483 fs_manager.cc:359] Metadata directory not provided
I20260809 12:20:25.545318 10483 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:20:25.545676 10483 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:20:25.548800 10483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "a4372730a7844930bf3bfbe08c4d4bae"
format_stamp: "Formatted at 2026-08-09 12:20:25 on dist-test-slave-39l8"
server_key: "08397aa2f3108341449c78ae979514b6"
server_key_iv: "52e6701d155b94d0388b3eb1541940b3"
server_key_version: "encryptionkey@0"
I20260809 12:20:25.549602 10483 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "a4372730a7844930bf3bfbe08c4d4bae"
format_stamp: "Formatted at 2026-08-09 12:20:25 on dist-test-slave-39l8"
server_key: "08397aa2f3108341449c78ae979514b6"
server_key_iv: "52e6701d155b94d0388b3eb1541940b3"
server_key_version: "encryptionkey@0"
I20260809 12:20:25.554862 10483 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 12:20:25.558187 10506 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:25.559510 10483 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:20:25.559693 10483 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "a4372730a7844930bf3bfbe08c4d4bae"
format_stamp: "Formatted at 2026-08-09 12:20:25 on dist-test-slave-39l8"
server_key: "08397aa2f3108341449c78ae979514b6"
server_key_iv: "52e6701d155b94d0388b3eb1541940b3"
server_key_version: "encryptionkey@0"
I20260809 12:20:25.559914 10483 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:20:25.580386 10483 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:20:25.584339 10483 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:20:25.584615 10483 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:20:25.603809 10483 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:39553
I20260809 12:20:25.603820 10589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:39553 every 8 connection(s)
I20260809 12:20:25.605156 10483 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:20:25.608831 10590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:20:25.609694 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 10483
I20260809 12:20:25.609929 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:20:25.610594 20191 external_mini_cluster.cc:1428] Setting key 22135088d93aa96b6eb65284bdbf3e9c
I20260809 12:20:25.623250 10590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae: Bootstrap starting.
Aug 09 12:20:25 dist-test-slave-39l8 krb5kdc[10458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:20:25.626992 10590 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae: Neither blocks nor log segments found. Creating new log.
I20260809 12:20:25.628094 10590 log.cc:824] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae: Log is configured to *not* fsync() on all Append() calls
I20260809 12:20:25.630950 10590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae: No bootstrap required, opened a new log
I20260809 12:20:25.634436 10594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:25.612701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47902 (local address 127.19.183.254:39553)
0809 12:20:25.613571 (+ 870us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:25.613583 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:25.613626 (+ 43us) server_negotiation.cc:407] Connection header received
0809 12:20:25.615336 (+ 1710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:25.615357 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:25.615754 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:25.616070 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:25.617913 (+ 1843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:25.619233 (+ 1320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:25.621302 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:25.621940 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:25.622074 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:25.627172 (+ 5098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:25.627181 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:25.627196 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:25.627232 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:25.630325 (+ 3093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:25.631079 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:25.631091 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:25.631098 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:25.631268 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:25.631668 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:25.631683 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:25.631685 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:25.632203 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:25.632535 (+ 332us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:25.633004 (+ 469us) server_negotiation.cc:299] Negotiation successful
0809 12:20:25.633443 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":650,"thread_start_us":446,"threads_started":1}
I20260809 12:20:25.639076 10590 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4372730a7844930bf3bfbe08c4d4bae" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39553 } }
I20260809 12:20:25.639506 10590 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:20:25.639631 10590 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4372730a7844930bf3bfbe08c4d4bae, State: Initialized, Role: FOLLOWER
I20260809 12:20:25.640215 10590 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4372730a7844930bf3bfbe08c4d4bae" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39553 } }
I20260809 12:20:25.640460 10590 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:20:25.640630 10590 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:20:25.640793 10590 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:20:25.642995 10590 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4372730a7844930bf3bfbe08c4d4bae" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39553 } }
I20260809 12:20:25.643524 10590 leader_election.cc:302] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4372730a7844930bf3bfbe08c4d4bae; no voters:
I20260809 12:20:25.644115 10590 leader_election.cc:288] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:20:25.644258 10597 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:20:25.645113 10597 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 1 LEADER]: Becoming Leader. State: Replica: a4372730a7844930bf3bfbe08c4d4bae, State: Running, Role: LEADER
I20260809 12:20:25.645663 10597 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4372730a7844930bf3bfbe08c4d4bae" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39553 } }
I20260809 12:20:25.646427 10590 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:20:25.648739 10598 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4372730a7844930bf3bfbe08c4d4bae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4372730a7844930bf3bfbe08c4d4bae" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39553 } } }
I20260809 12:20:25.648694 10601 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4372730a7844930bf3bfbe08c4d4bae. Latest consensus state: current_term: 1 leader_uuid: "a4372730a7844930bf3bfbe08c4d4bae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4372730a7844930bf3bfbe08c4d4bae" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39553 } } }
I20260809 12:20:25.649341 10598 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [sys.catalog]: This master's current role is: LEADER
I20260809 12:20:25.649353 10601 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [sys.catalog]: This master's current role is: LEADER
I20260809 12:20:25.651075 10606 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:20:25.657050 10606 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:20:25.666119 10606 catalog_manager.cc:1347] Generated new cluster ID: e120d0e86c994ebd97187a943dec94bc
I20260809 12:20:25.666244 10606 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:20:25.686745 10606 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:20:25.688067 10606 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:20:25.700595 10606 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae: Generated new TSK 0
I20260809 12:20:25.701730 10606 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
I20260809 12:20:25.793497 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:39553
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:20:26.176182 10629 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:20:26.176456 10629 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:20:26.176635 10629 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:20:26.183924 10629 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:20:26.184055 10629 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:20:26.184245 10629 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:20:26.193881 10629 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:39553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:20:26.195186 10629 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:20:26.196533 10629 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:20:26.209406 10639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:26.209484 10636 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:26.209493 10634 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:20:26.210613 10629 server_base.cc:1034] running on GCE node
I20260809 12:20:26.211241 10629 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:20:26.212181 10629 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:20:26.213402 10629 hybrid_clock.cc:648] HybridClock initialized: now 1786278026213319 us; error 76 us; skew 500 ppm
I20260809 12:20:26.213718 10629 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:20:26 dist-test-slave-39l8 krb5kdc[10458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278026, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:20:26.216847 10629 init.cc:377] Logged in from keytab as kudu/127.19.183.193@KRBTEST.COM (short username kudu)
I20260809 12:20:26.218298 10629 webserver.cc:460] Webserver started at http://127.19.183.193:39839/ using document root <none> and password file <none>
I20260809 12:20:26.219071 10629 fs_manager.cc:359] Metadata directory not provided
I20260809 12:20:26.219225 10629 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:20:26.219542 10629 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:20:26.222558 10629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "a1469d62f6084bec9ac860cc7d88741a"
format_stamp: "Formatted at 2026-08-09 12:20:26 on dist-test-slave-39l8"
server_key: "9eeff0c547456e0f543319953a68fb21"
server_key_iv: "77b6387c37d8de70616925c8fd2b65cb"
server_key_version: "encryptionkey@0"
I20260809 12:20:26.223414 10629 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "a1469d62f6084bec9ac860cc7d88741a"
format_stamp: "Formatted at 2026-08-09 12:20:26 on dist-test-slave-39l8"
server_key: "9eeff0c547456e0f543319953a68fb21"
server_key_iv: "77b6387c37d8de70616925c8fd2b65cb"
server_key_version: "encryptionkey@0"
I20260809 12:20:26.228547 10629 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:20:26.231910 10649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:26.233242 10629 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:20:26.233423 10629 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "a1469d62f6084bec9ac860cc7d88741a"
format_stamp: "Formatted at 2026-08-09 12:20:26 on dist-test-slave-39l8"
server_key: "9eeff0c547456e0f543319953a68fb21"
server_key_iv: "77b6387c37d8de70616925c8fd2b65cb"
server_key_version: "encryptionkey@0"
I20260809 12:20:26.233639 10629 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:20:26.254271 10629 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:20:26.258184 10629 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:20:26.258484 10629 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:20:26.259593 10629 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:20:26.260957 10629 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:20:26.261065 10629 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:26.261219 10629 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:20:26.261291 10629 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:26.295544 10629 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:43617
I20260809 12:20:26.295612 10811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:43617 every 8 connection(s)
I20260809 12:20:26.296797 10629 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:20:26.299052 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 10629
I20260809 12:20:26.299273 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:20:26.300176 20191 external_mini_cluster.cc:1428] Setting key b4c5daef6d6f44257e1933bf1042d10b
Aug 09 12:20:26 dist-test-slave-39l8 krb5kdc[10458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278026, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:20:26.320966 10817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:26.299572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:59193 (local address 127.19.183.254:39553)
0809 12:20:26.300142 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:26.300149 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:26.301153 (+ 1004us) server_negotiation.cc:407] Connection header received
0809 12:20:26.303641 (+ 2488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:26.303645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:26.303709 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:26.303802 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:26.305925 (+ 2123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:26.306613 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:26.309144 (+ 2531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:26.309527 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:26.309643 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:26.314347 (+ 4704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:26.314352 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:26.314355 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:26.314386 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:26.317609 (+ 3223us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:26.318308 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:26.318316 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:26.318323 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:26.318469 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:26.318946 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:26.318954 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:26.318960 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:26.319330 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:26.319493 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:26.320194 (+ 701us) server_negotiation.cc:299] Negotiation successful
0809 12:20:26.320466 (+ 272us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":256,"threads_started":1}
I20260809 12:20:26.321475 10818 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:26.300060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.193:59193)
0809 12:20:26.300967 (+ 907us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:26.301016 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:26.303358 (+ 2342us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:26.304082 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:26.304095 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:26.304935 (+ 840us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:20:26.305752 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:26.305780 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:26.306796 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:26.306804 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:26.308914 (+ 2110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:26.308933 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:26.309774 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:26.309784 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:26.310121 (+ 337us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:26.310954 (+ 833us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:20:26.310998 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:20:26.314130 (+ 3132us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:20:26.317750 (+ 3620us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:26.317757 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:26.317768 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:26.318175 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:26.318577 (+ 402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:26.318580 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:26.318582 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:26.318829 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:26.319459 (+ 630us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:20:26.319465 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:20:26.319772 (+ 307us) client_negotiation.cc:769] Sending connection context
0809 12:20:26.320000 (+ 228us) client_negotiation.cc:240] Negotiation successful
0809 12:20:26.320359 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":627,"thread_start_us":360,"threads_started":1}
I20260809 12:20:26.323117 10813 heartbeater.cc:347] Connected to a master server at 127.19.183.254:39553
I20260809 12:20:26.323395 10813 heartbeater.cc:464] Registering TS with master...
I20260809 12:20:26.324087 10813 heartbeater.cc:511] Master 127.19.183.254:39553 requested a full tablet report, sending...
I20260809 12:20:26.326153 10538 ts_manager.cc:194] Registered new tserver with Master: a1469d62f6084bec9ac860cc7d88741a (127.19.183.193:43617)
I20260809 12:20:26.327642 10538 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:59193
WARNING: no policy specified for kudu/127.19.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
I20260809 12:20:26.369475 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:39553
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:20:26.716507 10822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:20:26.716827 10822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:20:26.716988 10822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:20:26.724522 10822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:20:26.724671 10822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:20:26.724850 10822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:20:26.734933 10822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:39553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:20:26.736222 10822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:20:26.737571 10822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:20:26.755155 10832 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:26.755481 10833 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:26.755816 10836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:20:26.756668 10822 server_base.cc:1034] running on GCE node
I20260809 12:20:26.757272 10822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:20:26.758203 10822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:20:26.759447 10822 hybrid_clock.cc:648] HybridClock initialized: now 1786278026759401 us; error 50 us; skew 500 ppm
I20260809 12:20:26.759773 10822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:20:26 dist-test-slave-39l8 krb5kdc[10458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278026, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:20:26.762995 10822 init.cc:377] Logged in from keytab as kudu/127.19.183.194@KRBTEST.COM (short username kudu)
I20260809 12:20:26.764453 10822 webserver.cc:460] Webserver started at http://127.19.183.194:41643/ using document root <none> and password file <none>
I20260809 12:20:26.765147 10822 fs_manager.cc:359] Metadata directory not provided
I20260809 12:20:26.765290 10822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:20:26.765609 10822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:20:26.768675 10822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "45407bd46e4e4e4c94de4e814f066ede"
format_stamp: "Formatted at 2026-08-09 12:20:26 on dist-test-slave-39l8"
server_key: "cb5f28f572af396b9f51f43fbb331c0f"
server_key_iv: "177c99be4d213c1119edbb48e64b02d8"
server_key_version: "encryptionkey@0"
I20260809 12:20:26.769431 10822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "45407bd46e4e4e4c94de4e814f066ede"
format_stamp: "Formatted at 2026-08-09 12:20:26 on dist-test-slave-39l8"
server_key: "cb5f28f572af396b9f51f43fbb331c0f"
server_key_iv: "177c99be4d213c1119edbb48e64b02d8"
server_key_version: "encryptionkey@0"
I20260809 12:20:26.774559 10822 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 12:20:26.777923 10846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:26.779312 10822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:20:26.779481 10822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "45407bd46e4e4e4c94de4e814f066ede"
format_stamp: "Formatted at 2026-08-09 12:20:26 on dist-test-slave-39l8"
server_key: "cb5f28f572af396b9f51f43fbb331c0f"
server_key_iv: "177c99be4d213c1119edbb48e64b02d8"
server_key_version: "encryptionkey@0"
I20260809 12:20:26.779690 10822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:20:26.801543 10822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:20:26.805917 10822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:20:26.806226 10822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:20:26.807231 10822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:20:26.808519 10822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:20:26.808625 10822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:26.808768 10822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:20:26.808840 10822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:26.843158 10822 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:40951
I20260809 12:20:26.843217 11006 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:40951 every 8 connection(s)
I20260809 12:20:26.844543 10822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:20:26.850122 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 10822
I20260809 12:20:26.850334 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:20:26.850980 20191 external_mini_cluster.cc:1428] Setting key e17502df58851341b57bde1591193625
Aug 09 12:20:26 dist-test-slave-39l8 krb5kdc[10458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278026, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:20:26.867422 11011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:26.847132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:36029 (local address 127.19.183.254:39553)
0809 12:20:26.847602 (+ 470us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:26.847607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:26.848742 (+ 1135us) server_negotiation.cc:407] Connection header received
0809 12:20:26.851212 (+ 2470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:26.851217 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:26.851289 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:26.851385 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:26.853202 (+ 1817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:26.854258 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:26.855810 (+ 1552us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:26.856527 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:26.856676 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:26.860225 (+ 3549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:26.860233 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:26.860242 (+ 9us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:26.860289 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:26.863970 (+ 3681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:26.864809 (+ 839us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:26.864817 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:26.864823 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:26.864997 (+ 174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:26.865424 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:26.865431 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:26.865437 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:26.865830 (+ 393us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:26.865997 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:26.866680 (+ 683us) server_negotiation.cc:299] Negotiation successful
0809 12:20:26.866957 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":280,"threads_started":1}
I20260809 12:20:26.868304 11012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:26.847745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.194:36029)
0809 12:20:26.848592 (+ 847us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:26.848635 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:26.850875 (+ 2240us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:26.851701 (+ 826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:26.851714 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:26.852332 (+ 618us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:20:26.852993 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:26.853020 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:26.854402 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:26.854406 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:26.855658 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:26.855670 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:26.856723 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:26.856728 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:26.856920 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:26.857616 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:20:26.857639 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:20:26.859989 (+ 2350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:20:26.864141 (+ 4152us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:26.864148 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:26.864160 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:26.864628 (+ 468us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:26.865109 (+ 481us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:26.865112 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:26.865115 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:26.865297 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:26.865960 (+ 663us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:20:26.865965 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:20:26.866267 (+ 302us) client_negotiation.cc:769] Sending connection context
0809 12:20:26.866499 (+ 232us) client_negotiation.cc:240] Negotiation successful
0809 12:20:26.866936 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":611,"thread_start_us":302,"threads_started":1}
I20260809 12:20:26.869768 11007 heartbeater.cc:347] Connected to a master server at 127.19.183.254:39553
I20260809 12:20:26.870086 11007 heartbeater.cc:464] Registering TS with master...
I20260809 12:20:26.870851 11007 heartbeater.cc:511] Master 127.19.183.254:39553 requested a full tablet report, sending...
I20260809 12:20:26.872344 10538 ts_manager.cc:194] Registered new tserver with Master: 45407bd46e4e4e4c94de4e814f066ede (127.19.183.194:40951)
I20260809 12:20:26.873149 10538 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:36029
WARNING: no policy specified for kudu/127.19.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
I20260809 12:20:26.919323 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:39553
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:20:27.267398 11017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:20:27.267690 11017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:20:27.267894 11017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:20:27.275074 11017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:20:27.275182 11017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:20:27.275388 11017 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:20:27.285641 11017 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:39553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:20:27.286969 11017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:20:27.288323 11017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:20:27.301668 11030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:27.301792 11029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:27.301775 11033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:20:27.302759 11017 server_base.cc:1034] running on GCE node
I20260809 12:20:27.303383 11017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:20:27.304327 11017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:20:27.305560 11017 hybrid_clock.cc:648] HybridClock initialized: now 1786278027305511 us; error 49 us; skew 500 ppm
I20260809 12:20:27.305883 11017 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:20:27 dist-test-slave-39l8 krb5kdc[10458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278027, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:20:27.309165 11017 init.cc:377] Logged in from keytab as kudu/127.19.183.195@KRBTEST.COM (short username kudu)
I20260809 12:20:27.310580 11017 webserver.cc:460] Webserver started at http://127.19.183.195:39099/ using document root <none> and password file <none>
I20260809 12:20:27.311331 11017 fs_manager.cc:359] Metadata directory not provided
I20260809 12:20:27.311480 11017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:20:27.311801 11017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:20:27.314946 11017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "2006b09388984ccbadee161a62f02309"
format_stamp: "Formatted at 2026-08-09 12:20:27 on dist-test-slave-39l8"
server_key: "d914c256b31aaeee56b5cbe8b162867c"
server_key_iv: "89076bb2c71a8027af48ebe7127dbb41"
server_key_version: "encryptionkey@0"
I20260809 12:20:27.315758 11017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "2006b09388984ccbadee161a62f02309"
format_stamp: "Formatted at 2026-08-09 12:20:27 on dist-test-slave-39l8"
server_key: "d914c256b31aaeee56b5cbe8b162867c"
server_key_iv: "89076bb2c71a8027af48ebe7127dbb41"
server_key_version: "encryptionkey@0"
I20260809 12:20:27.321000 11017 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:20:27.324550 11044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:27.325971 11017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:20:27.326174 11017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "2006b09388984ccbadee161a62f02309"
format_stamp: "Formatted at 2026-08-09 12:20:27 on dist-test-slave-39l8"
server_key: "d914c256b31aaeee56b5cbe8b162867c"
server_key_iv: "89076bb2c71a8027af48ebe7127dbb41"
server_key_version: "encryptionkey@0"
I20260809 12:20:27.326383 11017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:20:27.330440 10813 heartbeater.cc:503] Master 127.19.183.254:39553 was elected leader, sending a full tablet report...
I20260809 12:20:27.343497 11017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:20:27.347772 11017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:20:27.348104 11017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:20:27.349061 11017 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:20:27.350420 11017 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:20:27.350494 11017 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:27.350687 11017 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:20:27.350800 11017 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
I20260809 12:20:27.385728 11017 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:41153
I20260809 12:20:27.385776 11208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:41153 every 8 connection(s)
I20260809 12:20:27.387123 11017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:20:27.390116 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 11017
I20260809 12:20:27.390333 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:20:27.391002 20191 external_mini_cluster.cc:1428] Setting key f33ee87c993084c47c9fe1c29b48ac56
Aug 09 12:20:27 dist-test-slave-39l8 krb5kdc[10458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278027, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:20:27.407714 11213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:27.389676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:59727 (local address 127.19.183.254:39553)
0809 12:20:27.390269 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:27.390276 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:27.391331 (+ 1055us) server_negotiation.cc:407] Connection header received
0809 12:20:27.393741 (+ 2410us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:27.393746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:27.393854 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:27.393979 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:27.395613 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.396394 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.397859 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.398222 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.398323 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:27.401716 (+ 3393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:27.401721 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:27.401725 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:27.401757 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:27.404246 (+ 2489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:27.405056 (+ 810us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:27.405061 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:27.405064 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:27.405242 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:27.405740 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:27.405745 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:27.405748 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:27.406123 (+ 375us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:27.406290 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:27.407074 (+ 784us) server_negotiation.cc:299] Negotiation successful
0809 12:20:27.407297 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":464,"thread_start_us":283,"threads_started":1}
I20260809 12:20:27.408708 11214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:27.390202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.195:59727)
0809 12:20:27.391143 (+ 941us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:27.391199 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:27.393479 (+ 2280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:27.394234 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:27.394245 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:27.394890 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:20:27.395448 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:27.395465 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.396528 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.396532 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:27.397741 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:27.397753 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.398416 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.398420 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:27.398646 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:27.399291 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:20:27.399313 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:20:27.401521 (+ 2208us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:20:27.404398 (+ 2877us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:27.404405 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:27.404417 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:27.404935 (+ 518us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:27.405389 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:27.405394 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:27.405399 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:27.405613 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:27.406256 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:20:27.406264 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:20:27.406646 (+ 382us) client_negotiation.cc:769] Sending connection context
0809 12:20:27.406967 (+ 321us) client_negotiation.cc:240] Negotiation successful
0809 12:20:27.407506 (+ 539us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":338,"threads_started":1}
I20260809 12:20:27.410079 11210 heartbeater.cc:347] Connected to a master server at 127.19.183.254:39553
I20260809 12:20:27.410348 11210 heartbeater.cc:464] Registering TS with master...
I20260809 12:20:27.411096 11210 heartbeater.cc:511] Master 127.19.183.254:39553 requested a full tablet report, sending...
I20260809 12:20:27.412539 10538 ts_manager.cc:194] Registered new tserver with Master: 2006b09388984ccbadee161a62f02309 (127.19.183.195:41153)
I20260809 12:20:27.413309 10538 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:59727
I20260809 12:20:27.418897 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:20:27.436825 11213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:27.423417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47918 (local address 127.19.183.254:39553)
0809 12:20:27.423580 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:27.423585 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:27.424051 (+ 466us) server_negotiation.cc:407] Connection header received
0809 12:20:27.424290 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:27.424294 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:27.424370 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:27.424472 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:27.425977 (+ 1505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.426804 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.428247 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.428680 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.428816 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:27.431494 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:27.431499 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:27.431502 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:27.431536 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:27.433952 (+ 2416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:27.434646 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:27.434655 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:27.434658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:27.434839 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:27.435234 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:27.435240 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:27.435243 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:27.435574 (+ 331us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:27.435828 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:27.436152 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 12:20:27.436394 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 12:20:27.440940 10538 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47918:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:20:27.448800 10538 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:20:27.473080 11234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:27.464884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36184 (local address 127.19.183.193:43617)
0809 12:20:27.465706 (+ 822us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:27.465715 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:27.465743 (+ 28us) server_negotiation.cc:407] Connection header received
0809 12:20:27.465845 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:27.465853 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:27.466090 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:27.466867 (+ 777us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:20:27.467109 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.468365 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.470600 (+ 2235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.471621 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.471828 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:27.472044 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:27.472557 (+ 513us) server_negotiation.cc:299] Negotiation successful
0809 12:20:27.472729 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":637,"thread_start_us":1277,"threads_started":1}
I20260809 12:20:27.473388 11233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:27.464384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43617 (local address 127.0.0.1:36184)
0809 12:20:27.465024 (+ 640us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:27.465042 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:27.465147 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:27.466304 (+ 1157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:27.466310 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:27.466335 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:20:27.466697 (+ 362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:27.466737 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.468499 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.468507 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:27.470438 (+ 1931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:27.470454 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.471827 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.471833 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:27.472284 (+ 451us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:27.472349 (+ 65us) client_negotiation.cc:769] Sending connection context
0809 12:20:27.472493 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 12:20:27.472702 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":499,"mutex_wait_us":23,"thread_start_us":746,"threads_started":1}
I20260809 12:20:27.473748 11231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:27.463149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38804 (local address 127.19.183.194:40951)
0809 12:20:27.463742 (+ 593us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:27.463755 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:27.464378 (+ 623us) server_negotiation.cc:407] Connection header received
0809 12:20:27.464638 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:27.464648 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:27.464879 (+ 231us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:27.465157 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:20:27.466177 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.467488 (+ 1311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.470692 (+ 3204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.472119 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.472266 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:27.472398 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:27.473265 (+ 867us) server_negotiation.cc:299] Negotiation successful
0809 12:20:27.473437 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":364,"thread_start_us":285,"threads_started":1}
I20260809 12:20:27.474455 11227 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:27.462912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:40951 (local address 127.0.0.1:38804)
0809 12:20:27.464254 (+ 1342us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:27.464275 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:27.464416 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:27.465419 (+ 1003us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:27.465429 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:27.465482 (+ 53us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:20:27.465973 (+ 491us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:27.465987 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.467628 (+ 1641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.467636 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:27.470515 (+ 2879us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:27.470528 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.472565 (+ 2037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.472573 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:27.472735 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:27.472766 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 12:20:27.472865 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 12:20:27.472993 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1207,"mutex_wait_us":459,"spinlock_wait_cycles":1280,"thread_start_us":355,"threads_started":1}
I20260809 12:20:27.475109 11229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:27.462154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43202 (local address 127.19.183.195:41153)
0809 12:20:27.463033 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:27.463042 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:27.463788 (+ 746us) server_negotiation.cc:407] Connection header received
0809 12:20:27.464309 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:27.464318 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:27.464587 (+ 269us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:27.464885 (+ 298us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:20:27.466013 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.467507 (+ 1494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.470557 (+ 3050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.472053 (+ 1496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.472238 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:27.472398 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:27.474560 (+ 2162us) server_negotiation.cc:299] Negotiation successful
0809 12:20:27.474762 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":695,"thread_start_us":485,"threads_started":1}
I20260809 12:20:27.475152 11230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:27.461901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:41153 (local address 127.0.0.1:43202)
0809 12:20:27.463633 (+ 1732us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:27.463674 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:27.463952 (+ 278us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:27.465235 (+ 1283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:27.465243 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:27.465273 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:20:27.465765 (+ 492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:27.465779 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.467685 (+ 1906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.467694 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:27.470324 (+ 2630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:27.470344 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.474079 (+ 3735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.474092 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:27.474313 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:27.474352 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 12:20:27.474512 (+ 160us) client_negotiation.cc:240] Negotiation successful
0809 12:20:27.474671 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1504,"mutex_wait_us":86,"thread_start_us":2029,"threads_started":1}
I20260809 12:20:27.477762 10720 tablet_service.cc:1467] Processing CreateTablet for tablet 03044428b615493fa8151e9067509f5b (DEFAULT_TABLE table=test-table [id=5f0725f1729f4e29b79cde88e6d5fc03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:20:27.478487 10917 tablet_service.cc:1467] Processing CreateTablet for tablet 03044428b615493fa8151e9067509f5b (DEFAULT_TABLE table=test-table [id=5f0725f1729f4e29b79cde88e6d5fc03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:20:27.478803 11107 tablet_service.cc:1467] Processing CreateTablet for tablet 03044428b615493fa8151e9067509f5b (DEFAULT_TABLE table=test-table [id=5f0725f1729f4e29b79cde88e6d5fc03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:20:27.479525 10720 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03044428b615493fa8151e9067509f5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:20:27.479903 10917 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03044428b615493fa8151e9067509f5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:20:27.480162 11107 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03044428b615493fa8151e9067509f5b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:20:27.492971 11238 tablet_bootstrap.cc:492] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309: Bootstrap starting.
I20260809 12:20:27.494549 11239 tablet_bootstrap.cc:492] T 03044428b615493fa8151e9067509f5b P 45407bd46e4e4e4c94de4e814f066ede: Bootstrap starting.
I20260809 12:20:27.495383 11241 tablet_bootstrap.cc:492] T 03044428b615493fa8151e9067509f5b P a1469d62f6084bec9ac860cc7d88741a: Bootstrap starting.
I20260809 12:20:27.498819 11238 tablet_bootstrap.cc:654] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309: Neither blocks nor log segments found. Creating new log.
I20260809 12:20:27.500252 11238 log.cc:824] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309: Log is configured to *not* fsync() on all Append() calls
I20260809 12:20:27.500752 11241 tablet_bootstrap.cc:654] T 03044428b615493fa8151e9067509f5b P a1469d62f6084bec9ac860cc7d88741a: Neither blocks nor log segments found. Creating new log.
I20260809 12:20:27.501981 11239 tablet_bootstrap.cc:654] T 03044428b615493fa8151e9067509f5b P 45407bd46e4e4e4c94de4e814f066ede: Neither blocks nor log segments found. Creating new log.
I20260809 12:20:27.502168 11241 log.cc:824] T 03044428b615493fa8151e9067509f5b P a1469d62f6084bec9ac860cc7d88741a: Log is configured to *not* fsync() on all Append() calls
I20260809 12:20:27.503238 11239 log.cc:824] T 03044428b615493fa8151e9067509f5b P 45407bd46e4e4e4c94de4e814f066ede: Log is configured to *not* fsync() on all Append() calls
I20260809 12:20:27.503837 11238 tablet_bootstrap.cc:492] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309: No bootstrap required, opened a new log
I20260809 12:20:27.504184 11238 ts_tablet_manager.cc:1397] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.004s
I20260809 12:20:27.505424 11241 tablet_bootstrap.cc:492] T 03044428b615493fa8151e9067509f5b P a1469d62f6084bec9ac860cc7d88741a: No bootstrap required, opened a new log
I20260809 12:20:27.505767 11241 ts_tablet_manager.cc:1397] T 03044428b615493fa8151e9067509f5b P a1469d62f6084bec9ac860cc7d88741a: Time spent bootstrapping tablet: real 0.011s user 0.005s sys 0.002s
I20260809 12:20:27.506408 11239 tablet_bootstrap.cc:492] T 03044428b615493fa8151e9067509f5b P 45407bd46e4e4e4c94de4e814f066ede: No bootstrap required, opened a new log
I20260809 12:20:27.506673 11239 ts_tablet_manager.cc:1397] T 03044428b615493fa8151e9067509f5b P 45407bd46e4e4e4c94de4e814f066ede: Time spent bootstrapping tablet: real 0.012s user 0.006s sys 0.000s
I20260809 12:20:27.512068 11238 raft_consensus.cc:348] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } } peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } }
I20260809 12:20:27.512490 11238 raft_consensus.cc:374] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:20:27.512650 11238 raft_consensus.cc:729] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2006b09388984ccbadee161a62f02309, State: Initialized, Role: FOLLOWER
I20260809 12:20:27.513280 11238 consensus_queue.cc:260] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } } peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } }
I20260809 12:20:27.514465 11210 heartbeater.cc:503] Master 127.19.183.254:39553 was elected leader, sending a full tablet report...
I20260809 12:20:27.515398 11238 ts_tablet_manager.cc:1428] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309: Time spent starting tablet: real 0.011s user 0.006s sys 0.004s
I20260809 12:20:27.517621 11239 raft_consensus.cc:348] T 03044428b615493fa8151e9067509f5b P 45407bd46e4e4e4c94de4e814f066ede [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } } peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } }
I20260809 12:20:27.518196 11239 raft_consensus.cc:374] T 03044428b615493fa8151e9067509f5b P 45407bd46e4e4e4c94de4e814f066ede [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:20:27.518348 11239 raft_consensus.cc:729] T 03044428b615493fa8151e9067509f5b P 45407bd46e4e4e4c94de4e814f066ede [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45407bd46e4e4e4c94de4e814f066ede, State: Initialized, Role: FOLLOWER
I20260809 12:20:27.519156 11241 raft_consensus.cc:348] T 03044428b615493fa8151e9067509f5b P a1469d62f6084bec9ac860cc7d88741a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } } peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } }
I20260809 12:20:27.519197 11239 consensus_queue.cc:260] T 03044428b615493fa8151e9067509f5b P 45407bd46e4e4e4c94de4e814f066ede [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } } peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } }
I20260809 12:20:27.519582 11241 raft_consensus.cc:374] T 03044428b615493fa8151e9067509f5b P a1469d62f6084bec9ac860cc7d88741a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:20:27.519707 11241 raft_consensus.cc:729] T 03044428b615493fa8151e9067509f5b P a1469d62f6084bec9ac860cc7d88741a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1469d62f6084bec9ac860cc7d88741a, State: Initialized, Role: FOLLOWER
I20260809 12:20:27.520332 11241 consensus_queue.cc:260] T 03044428b615493fa8151e9067509f5b P a1469d62f6084bec9ac860cc7d88741a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } } peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } }
I20260809 12:20:27.520852 11007 heartbeater.cc:503] Master 127.19.183.254:39553 was elected leader, sending a full tablet report...
I20260809 12:20:27.521713 11239 ts_tablet_manager.cc:1428] T 03044428b615493fa8151e9067509f5b P 45407bd46e4e4e4c94de4e814f066ede: Time spent starting tablet: real 0.015s user 0.014s sys 0.000s
I20260809 12:20:27.523257 11241 ts_tablet_manager.cc:1428] T 03044428b615493fa8151e9067509f5b P a1469d62f6084bec9ac860cc7d88741a: Time spent starting tablet: real 0.017s user 0.014s sys 0.001s
W20260809 12:20:27.549355 10815 tablet.cc:2369] T 03044428b615493fa8151e9067509f5b P a1469d62f6084bec9ac860cc7d88741a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:20:27.596566 11010 tablet.cc:2369] T 03044428b615493fa8151e9067509f5b P 45407bd46e4e4e4c94de4e814f066ede: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:20:27.638860 11211 tablet.cc:2369] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:20:27.788815 11245 raft_consensus.cc:482] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:20:27.789117 11245 raft_consensus.cc:504] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } } peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } }
I20260809 12:20:27.790942 11245 leader_election.cc:288] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a1469d62f6084bec9ac860cc7d88741a (127.19.183.193:43617), 45407bd46e4e4e4c94de4e814f066ede (127.19.183.194:40951)
I20260809 12:20:27.797786 11234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:27.791408 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:39633 (local address 127.19.183.193:43617)
0809 12:20:27.791645 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:27.791651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:27.791964 (+ 313us) server_negotiation.cc:407] Connection header received
0809 12:20:27.792257 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:27.792266 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:27.792354 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:27.792470 (+ 116us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:20:27.793015 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.793954 (+ 939us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.796085 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.796871 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.797044 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:27.797120 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:27.797398 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 12:20:27.797524 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 12:20:27.797859 11214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:27.791475 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43617 (local address 127.19.183.195:39633)
0809 12:20:27.791837 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:27.791859 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:27.792084 (+ 225us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:27.792513 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:27.792517 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:27.792646 (+ 129us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:20:27.792863 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:27.792880 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.794134 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.794140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:27.795905 (+ 1765us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:27.795934 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.797041 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.797047 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:27.797232 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:27.797257 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 12:20:27.797375 (+ 118us) client_negotiation.cc:240] Negotiation successful
0809 12:20:27.797511 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"mutex_wait_us":37,"spinlock_wait_cycles":24832}
I20260809 12:20:27.798069 11231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:27.791484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:57131 (local address 127.19.183.194:40951)
0809 12:20:27.791724 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:27.791734 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:27.792610 (+ 876us) server_negotiation.cc:407] Connection header received
0809 12:20:27.792737 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:27.792744 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:27.792826 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:27.792938 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:20:27.793552 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.794449 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.796421 (+ 1972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.797175 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.797348 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:27.797424 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:27.797709 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 12:20:27.797838 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 12:20:27.798470 11249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:27.791629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:40951 (local address 127.19.183.195:57131)
0809 12:20:27.792457 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:27.792475 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:27.792602 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:27.792965 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:27.792969 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:27.792994 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:20:27.793396 (+ 402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:27.793409 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.794596 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.794600 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:27.796266 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:27.796277 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.797384 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.797391 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:27.797539 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:27.797562 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 12:20:27.797668 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 12:20:27.797796 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":691,"mutex_wait_us":47,"thread_start_us":451,"threads_started":1}
I20260809 12:20:27.798955 10749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03044428b615493fa8151e9067509f5b" candidate_uuid: "2006b09388984ccbadee161a62f02309" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1469d62f6084bec9ac860cc7d88741a" is_pre_election: true
I20260809 12:20:27.799484 10749 raft_consensus.cc:2454] T 03044428b615493fa8151e9067509f5b P a1469d62f6084bec9ac860cc7d88741a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2006b09388984ccbadee161a62f02309 in term 0.
I20260809 12:20:27.799474 10946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03044428b615493fa8151e9067509f5b" candidate_uuid: "2006b09388984ccbadee161a62f02309" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45407bd46e4e4e4c94de4e814f066ede" is_pre_election: true
I20260809 12:20:27.800030 10946 raft_consensus.cc:2454] T 03044428b615493fa8151e9067509f5b P 45407bd46e4e4e4c94de4e814f066ede [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2006b09388984ccbadee161a62f02309 in term 0.
I20260809 12:20:27.800227 11045 leader_election.cc:302] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2006b09388984ccbadee161a62f02309, a1469d62f6084bec9ac860cc7d88741a; no voters:
I20260809 12:20:27.800647 11245 raft_consensus.cc:2781] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:20:27.800807 11245 raft_consensus.cc:482] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:20:27.800918 11245 raft_consensus.cc:3037] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:20:27.803206 11245 raft_consensus.cc:504] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } } peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } }
I20260809 12:20:27.803889 11245 leader_election.cc:288] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309 [CANDIDATE]: Term 1 election: Requested vote from peers a1469d62f6084bec9ac860cc7d88741a (127.19.183.193:43617), 45407bd46e4e4e4c94de4e814f066ede (127.19.183.194:40951)
I20260809 12:20:27.804306 10749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03044428b615493fa8151e9067509f5b" candidate_uuid: "2006b09388984ccbadee161a62f02309" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a1469d62f6084bec9ac860cc7d88741a"
I20260809 12:20:27.804415 10946 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03044428b615493fa8151e9067509f5b" candidate_uuid: "2006b09388984ccbadee161a62f02309" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45407bd46e4e4e4c94de4e814f066ede"
I20260809 12:20:27.804551 10749 raft_consensus.cc:3037] T 03044428b615493fa8151e9067509f5b P a1469d62f6084bec9ac860cc7d88741a [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:20:27.804638 10946 raft_consensus.cc:3037] T 03044428b615493fa8151e9067509f5b P 45407bd46e4e4e4c94de4e814f066ede [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:20:27.807981 10749 raft_consensus.cc:2454] T 03044428b615493fa8151e9067509f5b P a1469d62f6084bec9ac860cc7d88741a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2006b09388984ccbadee161a62f02309 in term 1.
I20260809 12:20:27.807981 10946 raft_consensus.cc:2454] T 03044428b615493fa8151e9067509f5b P 45407bd46e4e4e4c94de4e814f066ede [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2006b09388984ccbadee161a62f02309 in term 1.
I20260809 12:20:27.808465 11046 leader_election.cc:302] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2006b09388984ccbadee161a62f02309, 45407bd46e4e4e4c94de4e814f066ede; no voters:
I20260809 12:20:27.808785 11245 raft_consensus.cc:2781] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:20:27.809306 11245 raft_consensus.cc:686] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309 [term 1 LEADER]: Becoming Leader. State: Replica: 2006b09388984ccbadee161a62f02309, State: Running, Role: LEADER
I20260809 12:20:27.809870 11245 consensus_queue.cc:237] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } } peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } }
I20260809 12:20:27.814360 10538 catalog_manager.cc:5495] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2006b09388984ccbadee161a62f02309 (127.19.183.195). New cstate: current_term: 1 leader_uuid: "2006b09388984ccbadee161a62f02309" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } health_report { overall_health: UNKNOWN } } }
I20260809 12:20:27.843679 11229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:27.838216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43206 (local address 127.19.183.195:41153)
0809 12:20:27.838386 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:27.838391 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:27.838525 (+ 134us) server_negotiation.cc:407] Connection header received
0809 12:20:27.838743 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:27.838748 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:27.838838 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:27.838966 (+ 128us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:20:27.839345 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.840033 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.841519 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.841912 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.842002 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:27.842521 (+ 519us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:20:27.843056 (+ 535us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:20:27.843220 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:27.843313 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 12:20:27.843394 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 12:20:27.852255 10749 raft_consensus.cc:1261] T 03044428b615493fa8151e9067509f5b P a1469d62f6084bec9ac860cc7d88741a [term 1 FOLLOWER]: Refusing update from remote peer 2006b09388984ccbadee161a62f02309: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:20:27.852330 10946 raft_consensus.cc:1261] T 03044428b615493fa8151e9067509f5b P 45407bd46e4e4e4c94de4e814f066ede [term 1 FOLLOWER]: Refusing update from remote peer 2006b09388984ccbadee161a62f02309: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:20:27.853284 11250 consensus_queue.cc:1035] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:20:27.853734 11245 consensus_queue.cc:1035] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:20:27.865425 11257 mvcc.cc:205] Tried to move back new op lower bound from 7316594802071961600 to 7316594801913294848. Current Snapshot: MvccSnapshot[applied={T|T < 7316594802071961600}]
I20260809 12:20:27.880594 10537 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:20:27.880882 10537 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47918:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:20:27.884691 10537 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae: Sending DeleteTablet for 3 replicas of tablet 03044428b615493fa8151e9067509f5b
I20260809 12:20:27.885952 10917 tablet_service.cc:1514] Processing DeleteTablet for tablet 03044428b615493fa8151e9067509f5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:20:27 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:38804
I20260809 12:20:27.885952 10720 tablet_service.cc:1514] Processing DeleteTablet for tablet 03044428b615493fa8151e9067509f5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:20:27 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:36184
I20260809 12:20:27.886180 11107 tablet_service.cc:1514] Processing DeleteTablet for tablet 03044428b615493fa8151e9067509f5b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:20:27 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:43202
I20260809 12:20:27.887122 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 10629
I20260809 12:20:27.901808 11277 tablet_replica.cc:330] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309: stopping tablet replica
I20260809 12:20:27.902824 11274 tablet_replica.cc:330] T 03044428b615493fa8151e9067509f5b P 45407bd46e4e4e4c94de4e814f066ede: stopping tablet replica
I20260809 12:20:27.903309 11274 raft_consensus.cc:2229] T 03044428b615493fa8151e9067509f5b P 45407bd46e4e4e4c94de4e814f066ede [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:20:27.903324 11277 raft_consensus.cc:2229] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:20:27.903861 11274 raft_consensus.cc:2258] T 03044428b615493fa8151e9067509f5b P 45407bd46e4e4e4c94de4e814f066ede [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:20:27.904604 11277 raft_consensus.cc:2258] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:20:27.904856 11272 tablet_replica.cc:330] T 03044428b615493fa8151e9067509f5b P a1469d62f6084bec9ac860cc7d88741a: stopping tablet replica
I20260809 12:20:27.905356 11272 raft_consensus.cc:2229] T 03044428b615493fa8151e9067509f5b P a1469d62f6084bec9ac860cc7d88741a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:20:27.905930 11272 raft_consensus.cc:2258] T 03044428b615493fa8151e9067509f5b P a1469d62f6084bec9ac860cc7d88741a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:20:27.906380 11274 ts_tablet_manager.cc:1905] T 03044428b615493fa8151e9067509f5b P 45407bd46e4e4e4c94de4e814f066ede: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:20:27.907090 11277 ts_tablet_manager.cc:1905] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:20:27.908012 11272 ts_tablet_manager.cc:1905] T 03044428b615493fa8151e9067509f5b P a1469d62f6084bec9ac860cc7d88741a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:20:27.913735 11274 ts_tablet_manager.cc:1918] T 03044428b615493fa8151e9067509f5b P 45407bd46e4e4e4c94de4e814f066ede: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
Aug 09 12:20:27 dist-test-slave-39l8 krb5kdc[10458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.193@KRBTEST.COM
I20260809 12:20:27.914951 11277 ts_tablet_manager.cc:1918] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:20:27.915068 11272 ts_tablet_manager.cc:1918] T 03044428b615493fa8151e9067509f5b P a1469d62f6084bec9ac860cc7d88741a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:20:27.915259 11277 log.cc:1196] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/03044428b615493fa8151e9067509f5b
I20260809 12:20:27.915297 11272 log.cc:1196] T 03044428b615493fa8151e9067509f5b P a1469d62f6084bec9ac860cc7d88741a: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/03044428b615493fa8151e9067509f5b
I20260809 12:20:27.915912 11277 ts_tablet_manager.cc:1939] T 03044428b615493fa8151e9067509f5b P 2006b09388984ccbadee161a62f02309: Deleting consensus metadata
I20260809 12:20:27.916095 11272 ts_tablet_manager.cc:1939] T 03044428b615493fa8151e9067509f5b P a1469d62f6084bec9ac860cc7d88741a: Deleting consensus metadata
I20260809 12:20:27.918205 10508 catalog_manager.cc:4841] TS a1469d62f6084bec9ac860cc7d88741a (127.19.183.193:43617): tablet 03044428b615493fa8151e9067509f5b (table test-table [id=5f0725f1729f4e29b79cde88e6d5fc03]) successfully deleted
I20260809 12:20:27.918293 11274 log.cc:1196] T 03044428b615493fa8151e9067509f5b P 45407bd46e4e4e4c94de4e814f066ede: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/03044428b615493fa8151e9067509f5b
I20260809 12:20:27.918603 10508 catalog_manager.cc:4841] TS 2006b09388984ccbadee161a62f02309 (127.19.183.195:41153): tablet 03044428b615493fa8151e9067509f5b (table test-table [id=5f0725f1729f4e29b79cde88e6d5fc03]) successfully deleted
I20260809 12:20:27.918902 11274 ts_tablet_manager.cc:1939] T 03044428b615493fa8151e9067509f5b P 45407bd46e4e4e4c94de4e814f066ede: Deleting consensus metadata
I20260809 12:20:27.920802 10509 catalog_manager.cc:4841] TS 45407bd46e4e4e4c94de4e814f066ede (127.19.183.194:40951): tablet 03044428b615493fa8151e9067509f5b (table test-table [id=5f0725f1729f4e29b79cde88e6d5fc03]) successfully deleted
I20260809 12:20:27.923283 11234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:27.888625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36196 (local address 127.19.183.193:43617)
0809 12:20:27.888865 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:27.888874 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:27.902130 (+ 13256us) server_negotiation.cc:407] Connection header received
0809 12:20:27.902366 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:27.902374 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:27.902489 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:27.902625 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:27.904990 (+ 2365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.906249 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.908408 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:27.909085 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:27.909194 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:27.915618 (+ 6424us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:27.915642 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:27.915682 (+ 40us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:27.915742 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:27.919966 (+ 4224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:27.920881 (+ 915us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:27.920890 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:27.920894 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:27.921092 (+ 198us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:27.921532 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:27.921537 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:27.921540 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:27.921939 (+ 399us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:27.922270 (+ 331us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:27.922599 (+ 329us) server_negotiation.cc:299] Negotiation successful
0809 12:20:27.922865 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 12:20:27.923985 10806 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:20:28.070547 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 10629
I20260809 12:20:28.108217 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 10822
Aug 09 12:20:28 dist-test-slave-39l8 krb5kdc[10458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.194@KRBTEST.COM
I20260809 12:20:28.132086 11231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:28.109129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38806 (local address 127.19.183.194:40951)
0809 12:20:28.109329 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:28.109335 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:28.109360 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:20:28.109440 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:28.109447 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:28.109544 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:28.109665 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:28.111119 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:28.112260 (+ 1141us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:28.113703 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:28.114386 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:28.114531 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:28.124667 (+ 10136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:28.124691 (+ 24us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:28.124714 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:28.124777 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:28.128577 (+ 3800us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:28.129549 (+ 972us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:28.129559 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:28.129563 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:28.129750 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:28.130124 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:28.130129 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:28.130132 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:28.130637 (+ 505us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:28.131153 (+ 516us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:28.131418 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 12:20:28.131653 (+ 235us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 12:20:28.132902 10997 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:20:28.225112 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 10822
I20260809 12:20:28.250391 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 11017
Aug 09 12:20:28 dist-test-slave-39l8 krb5kdc[10458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278025, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.195@KRBTEST.COM
I20260809 12:20:28.268352 11229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:28.251228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43222 (local address 127.19.183.195:41153)
0809 12:20:28.251450 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:28.251457 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:28.251591 (+ 134us) server_negotiation.cc:407] Connection header received
0809 12:20:28.251822 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:28.251833 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:28.251923 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:28.252059 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:28.253468 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:28.254283 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:28.255822 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:28.256275 (+ 453us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:28.256427 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:28.261172 (+ 4745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:28.261190 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:28.261225 (+ 35us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:28.261271 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:28.265336 (+ 4065us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:28.266012 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:28.266021 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:28.266025 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:28.266249 (+ 224us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:28.266594 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:28.266599 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:28.266602 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:28.267142 (+ 540us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:28.267495 (+ 353us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:28.267655 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 12:20:28.267897 (+ 242us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 12:20:28.269167 11204 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:20:28.393319 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 11017
I20260809 12:20:28.423621 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 10483
I20260809 12:20:28.424796 10582 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:20:28.539673 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 10483
I20260809 12:20:48.570704 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:39553
--webserver_interface=127.19.183.254
--webserver_port=43151
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:39553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:20:48.914747 11287 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:20:48.915055 11287 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:20:48.915235 11287 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:20:48.922621 11287 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 12:20:48.922761 11287 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 12:20:48.922853 11287 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 12:20:48.922945 11287 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:20:48.923039 11287 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:20:48.923154 11287 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:20:48.923229 11287 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:20:48.923306 11287 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:20:48.933314 11287 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:39553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:39553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=43151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:20:48.934559 11287 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:20:48.936278 11287 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:20:48.963236 11295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:48.963236 11296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:48.963289 11298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:20:48.963987 11287 server_base.cc:1034] running on GCE node
I20260809 12:20:48.964864 11287 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:20:48.966362 11287 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:20:48.967610 11287 hybrid_clock.cc:648] HybridClock initialized: now 1786278048967535 us; error 59 us; skew 500 ppm
I20260809 12:20:48.967985 11287 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:20:48 dist-test-slave-39l8 krb5kdc[10458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278048, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:20:48.971786 11287 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:20:48.973549 11287 webserver.cc:460] Webserver started at http://127.19.183.254:43151/ using document root <none> and password file <none>
I20260809 12:20:48.974346 11287 fs_manager.cc:359] Metadata directory not provided
I20260809 12:20:48.974490 11287 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:20:48.980516 11287 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260809 12:20:48.983536 11312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:48.984910 11287 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 12:20:48.985090 11287 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "a4372730a7844930bf3bfbe08c4d4bae"
format_stamp: "Formatted at 2026-08-09 12:20:25 on dist-test-slave-39l8"
server_key: "08397aa2f3108341449c78ae979514b6"
server_key_iv: "52e6701d155b94d0388b3eb1541940b3"
server_key_version: "encryptionkey@0"
I20260809 12:20:48.986136 11287 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:20:49.005720 11287 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:20:49.009974 11287 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:20:49.010316 11287 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:20:49.030575 11287 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:39553
I20260809 12:20:49.030584 11388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:39553 every 8 connection(s)
I20260809 12:20:49.032181 11287 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:20:49.034003 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 11287
I20260809 12:20:49.034977 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:43617
--local_ip_for_outbound_sockets=127.19.183.193
--tserver_master_addrs=127.19.183.254:39553
--webserver_port=39839
--webserver_interface=127.19.183.193
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:20:49.039469 11390 sys_catalog.cc:263] Verifying existing consensus state
I20260809 12:20:49.042867 11390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae: Bootstrap starting.
I20260809 12:20:49.068540 11390 log.cc:824] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae: Log is configured to *not* fsync() on all Append() calls
I20260809 12:20:49.085132 11390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:20:49.085789 11390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae: Bootstrap complete.
I20260809 12:20:49.099561 11390 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4372730a7844930bf3bfbe08c4d4bae" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39553 } }
I20260809 12:20:49.100737 11390 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4372730a7844930bf3bfbe08c4d4bae, State: Initialized, Role: FOLLOWER
I20260809 12:20:49.101475 11390 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4372730a7844930bf3bfbe08c4d4bae" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39553 } }
I20260809 12:20:49.101778 11390 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:20:49.101927 11390 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:20:49.102103 11390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 1 FOLLOWER]: Advancing to term 2
I20260809 12:20:49.108037 11390 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4372730a7844930bf3bfbe08c4d4bae" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39553 } }
I20260809 12:20:49.108651 11390 leader_election.cc:302] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4372730a7844930bf3bfbe08c4d4bae; no voters:
I20260809 12:20:49.109345 11390 leader_election.cc:288] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 12:20:49.109501 11395 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 2 FOLLOWER]: Leader election won for term 2
I20260809 12:20:49.110927 11395 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 2 LEADER]: Becoming Leader. State: Replica: a4372730a7844930bf3bfbe08c4d4bae, State: Running, Role: LEADER
I20260809 12:20:49.111567 11390 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:20:49.111464 11395 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4372730a7844930bf3bfbe08c4d4bae" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39553 } }
I20260809 12:20:49.114034 11396 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a4372730a7844930bf3bfbe08c4d4bae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4372730a7844930bf3bfbe08c4d4bae" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39553 } } }
I20260809 12:20:49.114421 11396 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [sys.catalog]: This master's current role is: LEADER
I20260809 12:20:49.118321 11398 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4372730a7844930bf3bfbe08c4d4bae. Latest consensus state: current_term: 2 leader_uuid: "a4372730a7844930bf3bfbe08c4d4bae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4372730a7844930bf3bfbe08c4d4bae" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39553 } } }
I20260809 12:20:49.119092 11398 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [sys.catalog]: This master's current role is: LEADER
I20260809 12:20:49.120810 11404 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:20:49.128829 11404 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:20:49.130264 11404 catalog_manager.cc:1259] Loaded cluster ID: e120d0e86c994ebd97187a943dec94bc
I20260809 12:20:49.130400 11404 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:20:49.134428 11404 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:20:49.136149 11404 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae: Loaded TSK: 0
I20260809 12:20:49.171146 11404 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae: Generated new TSK 1
I20260809 12:20:49.171599 11404 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 12:20:49.423187 11393 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:20:49.423452 11393 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:20:49.423607 11393 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:20:49.430821 11393 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:20:49.430931 11393 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:20:49.431124 11393 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:20:49.440797 11393 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:43617
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=39839
--tserver_master_addrs=127.19.183.254:39553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:20:49.442101 11393 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:20:49.443465 11393 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:20:49.456449 11437 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:49.456950 11436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:49.457059 11442 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:20:49.458935 11393 server_base.cc:1034] running on GCE node
I20260809 12:20:49.459627 11393 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:20:49.460659 11393 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:20:49.461876 11393 hybrid_clock.cc:648] HybridClock initialized: now 1786278049461826 us; error 43 us; skew 500 ppm
I20260809 12:20:49.462188 11393 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:20:49 dist-test-slave-39l8 krb5kdc[10458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278049, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:20:49.465353 11393 init.cc:377] Logged in from keytab as kudu/127.19.183.193@KRBTEST.COM (short username kudu)
I20260809 12:20:49.466909 11393 webserver.cc:460] Webserver started at http://127.19.183.193:39839/ using document root <none> and password file <none>
I20260809 12:20:49.467621 11393 fs_manager.cc:359] Metadata directory not provided
I20260809 12:20:49.467777 11393 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:20:49.473400 11393 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.002s sys 0.004s
I20260809 12:20:49.476195 11452 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:49.477567 11393 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:20:49.477739 11393 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "a1469d62f6084bec9ac860cc7d88741a"
format_stamp: "Formatted at 2026-08-09 12:20:26 on dist-test-slave-39l8"
server_key: "9eeff0c547456e0f543319953a68fb21"
server_key_iv: "77b6387c37d8de70616925c8fd2b65cb"
server_key_version: "encryptionkey@0"
I20260809 12:20:49.478667 11393 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:20:49.491400 11393 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:20:49.495487 11393 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:20:49.495757 11393 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:20:49.496685 11393 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:20:49.497993 11393 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:20:49.498122 11393 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:49.498250 11393 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:20:49.498346 11393 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:49.532173 11393 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:43617
I20260809 12:20:49.532265 11631 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:43617 every 8 connection(s)
I20260809 12:20:49.533703 11393 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:20:49.543527 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 11393
I20260809 12:20:49.544329 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:40951
--local_ip_for_outbound_sockets=127.19.183.194
--tserver_master_addrs=127.19.183.254:39553
--webserver_port=41643
--webserver_interface=127.19.183.194
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:20:49 dist-test-slave-39l8 krb5kdc[10458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278049, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:20:49.558295 11636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:49.536523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:32855 (local address 127.19.183.254:39553)
0809 12:20:49.537168 (+ 645us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:49.537189 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:49.537747 (+ 558us) server_negotiation.cc:407] Connection header received
0809 12:20:49.539895 (+ 2148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:49.539907 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:49.540214 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:49.540504 (+ 290us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:49.541910 (+ 1406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:49.543223 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:49.545635 (+ 2412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:49.546251 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:49.546376 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:49.550874 (+ 4498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:49.550881 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:49.550894 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:49.550931 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:49.553822 (+ 2891us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:49.554809 (+ 987us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:49.554816 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:49.554821 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:49.554975 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:49.555461 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:49.555464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:49.555466 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:49.555902 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:49.556054 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:49.556943 (+ 889us) server_negotiation.cc:299] Negotiation successful
0809 12:20:49.557244 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":402,"thread_start_us":270,"threads_started":1}
I20260809 12:20:49.558696 11640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:49.536818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.193:32855)
0809 12:20:49.537561 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:49.537608 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:49.539419 (+ 1811us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:49.540676 (+ 1257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:49.540684 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:49.541197 (+ 513us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:20:49.541728 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:49.541745 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:49.543412 (+ 1667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:49.543420 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:49.545474 (+ 2054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:49.545500 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:49.546483 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:49.546492 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:49.546828 (+ 336us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:49.547554 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:20:49.547587 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:20:49.550504 (+ 2917us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:20:49.553974 (+ 3470us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:49.553985 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:49.554002 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:49.554637 (+ 635us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:49.555091 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:49.555096 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:49.555101 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:49.555346 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:49.556039 (+ 693us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:20:49.556049 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:20:49.556458 (+ 409us) client_negotiation.cc:769] Sending connection context
0809 12:20:49.556763 (+ 305us) client_negotiation.cc:240] Negotiation successful
0809 12:20:49.557318 (+ 555us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":503,"thread_start_us":304,"threads_started":1}
I20260809 12:20:49.561576 11632 heartbeater.cc:347] Connected to a master server at 127.19.183.254:39553
I20260809 12:20:49.561885 11632 heartbeater.cc:464] Registering TS with master...
I20260809 12:20:49.562602 11632 heartbeater.cc:511] Master 127.19.183.254:39553 requested a full tablet report, sending...
I20260809 12:20:49.564783 11336 ts_manager.cc:194] Registered new tserver with Master: a1469d62f6084bec9ac860cc7d88741a (127.19.183.193:43617)
I20260809 12:20:49.566298 11336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:32855
W20260809 12:20:49.891295 11642 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:20:49.891634 11642 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:20:49.891821 11642 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:20:49.902525 11642 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:20:49.902661 11642 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:20:49.902885 11642 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:20:49.915798 11642 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:40951
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=41643
--tserver_master_addrs=127.19.183.254:39553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:20:49.917162 11642 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:20:49.918499 11642 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:20:49.932332 11647 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:49.932304 11648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:49.932237 11651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:20:49.933060 11642 server_base.cc:1034] running on GCE node
I20260809 12:20:49.933629 11642 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:20:49.934535 11642 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:20:49.935732 11642 hybrid_clock.cc:648] HybridClock initialized: now 1786278049935656 us; error 64 us; skew 500 ppm
I20260809 12:20:49.936002 11642 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:20:49 dist-test-slave-39l8 krb5kdc[10458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278049, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:20:49.939164 11642 init.cc:377] Logged in from keytab as kudu/127.19.183.194@KRBTEST.COM (short username kudu)
I20260809 12:20:49.940557 11642 webserver.cc:460] Webserver started at http://127.19.183.194:41643/ using document root <none> and password file <none>
I20260809 12:20:49.941228 11642 fs_manager.cc:359] Metadata directory not provided
I20260809 12:20:49.941349 11642 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:20:49.946512 11642 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:20:49.949219 11659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:49.950474 11642 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:20:49.950626 11642 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "45407bd46e4e4e4c94de4e814f066ede"
format_stamp: "Formatted at 2026-08-09 12:20:26 on dist-test-slave-39l8"
server_key: "cb5f28f572af396b9f51f43fbb331c0f"
server_key_iv: "177c99be4d213c1119edbb48e64b02d8"
server_key_version: "encryptionkey@0"
I20260809 12:20:49.951574 11642 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:20:49.977200 11642 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:20:49.981405 11642 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:20:49.981637 11642 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:20:49.982592 11642 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:20:49.983901 11642 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:20:49.984025 11642 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:49.984160 11642 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:20:49.984252 11642 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:50.023645 11642 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:40951
I20260809 12:20:50.023705 11820 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:40951 every 8 connection(s)
I20260809 12:20:50.025097 11642 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:20:50.028137 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 11642
I20260809 12:20:50.028797 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:41153
--local_ip_for_outbound_sockets=127.19.183.195
--tserver_master_addrs=127.19.183.254:39553
--webserver_port=39099
--webserver_interface=127.19.183.195
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 12:20:50 dist-test-slave-39l8 krb5kdc[10458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278049, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:20:50.051467 11636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:50.027681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:40325 (local address 127.19.183.254:39553)
0809 12:20:50.027929 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:50.027936 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:50.029401 (+ 1465us) server_negotiation.cc:407] Connection header received
0809 12:20:50.032048 (+ 2647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:50.032056 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:50.032187 (+ 131us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:50.032305 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:50.034299 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.035488 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.037734 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.038541 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.038726 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:50.043354 (+ 4628us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:50.043362 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:50.043368 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:50.043419 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:50.047683 (+ 4264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:50.048660 (+ 977us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:50.048669 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:50.048675 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:50.048889 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:50.049420 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:50.049428 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:50.049433 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:50.049850 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:50.050010 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:50.050740 (+ 730us) server_negotiation.cc:299] Negotiation successful
0809 12:20:50.051020 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260809 12:20:50.052297 11825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:50.028168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.194:40325)
0809 12:20:50.029229 (+ 1061us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:50.029281 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:50.031721 (+ 2440us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:50.032582 (+ 861us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:50.032602 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:50.033278 (+ 676us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:20:50.034042 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:50.034068 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.035666 (+ 1598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.035671 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:50.037563 (+ 1892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:50.037578 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.038883 (+ 1305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.038890 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:50.039192 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:50.040003 (+ 811us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:20:50.040041 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:20:50.043028 (+ 2987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:20:50.047849 (+ 4821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:50.047859 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:50.047875 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:50.048509 (+ 634us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:50.049024 (+ 515us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:50.049029 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:50.049032 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:50.049279 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:50.049993 (+ 714us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:20:50.049999 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:20:50.050309 (+ 310us) client_negotiation.cc:769] Sending connection context
0809 12:20:50.050541 (+ 232us) client_negotiation.cc:240] Negotiation successful
0809 12:20:50.051024 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":618,"thread_start_us":346,"threads_started":1}
I20260809 12:20:50.053840 11821 heartbeater.cc:347] Connected to a master server at 127.19.183.254:39553
I20260809 12:20:50.054195 11821 heartbeater.cc:464] Registering TS with master...
I20260809 12:20:50.055111 11821 heartbeater.cc:511] Master 127.19.183.254:39553 requested a full tablet report, sending...
I20260809 12:20:50.056876 11336 ts_manager.cc:194] Registered new tserver with Master: 45407bd46e4e4e4c94de4e814f066ede (127.19.183.194:40951)
I20260809 12:20:50.058198 11336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:40325
W20260809 12:20:50.373698 11827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:20:50.373950 11827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:20:50.374043 11827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:20:50.380976 11827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:20:50.381068 11827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:20:50.381167 11827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:20:50.390520 11827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:41153
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=39099
--tserver_master_addrs=127.19.183.254:39553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:20:50.391749 11827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:20:50.392988 11827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:20:50.405771 11840 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:50.405895 11845 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:20:50.406221 11839 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:20:50.407021 11827 server_base.cc:1034] running on GCE node
I20260809 12:20:50.407588 11827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:20:50.408449 11827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:20:50.409694 11827 hybrid_clock.cc:648] HybridClock initialized: now 1786278050409608 us; error 70 us; skew 500 ppm
I20260809 12:20:50.410023 11827 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:20:50 dist-test-slave-39l8 krb5kdc[10458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278050, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:20:50.418637 11827 init.cc:377] Logged in from keytab as kudu/127.19.183.195@KRBTEST.COM (short username kudu)
I20260809 12:20:50.420230 11827 webserver.cc:460] Webserver started at http://127.19.183.195:39099/ using document root <none> and password file <none>
I20260809 12:20:50.420992 11827 fs_manager.cc:359] Metadata directory not provided
I20260809 12:20:50.421171 11827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:20:50.426472 11827 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:20:50.429339 11855 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:50.430658 11827 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:20:50.430855 11827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "2006b09388984ccbadee161a62f02309"
format_stamp: "Formatted at 2026-08-09 12:20:27 on dist-test-slave-39l8"
server_key: "d914c256b31aaeee56b5cbe8b162867c"
server_key_iv: "89076bb2c71a8027af48ebe7127dbb41"
server_key_version: "encryptionkey@0"
I20260809 12:20:50.431808 11827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:20:50.460381 11827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:20:50.464248 11827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:20:50.464558 11827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:20:50.465556 11827 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:20:50.466889 11827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:20:50.467005 11827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:50.467144 11827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:20:50.467235 11827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:20:50.500252 11827 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:41153
I20260809 12:20:50.500334 12025 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:41153 every 8 connection(s)
I20260809 12:20:50.501670 11827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:20:50.506318 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 11827
Aug 09 12:20:50 dist-test-slave-39l8 krb5kdc[10458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278050, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:20:50.524427 12033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:50.507278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44868 (local address 127.19.183.254:39553)
0809 12:20:50.508079 (+ 801us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:50.508083 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:50.508105 (+ 22us) server_negotiation.cc:407] Connection header received
0809 12:20:50.508147 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:50.508150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:50.508218 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:50.508316 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:50.510335 (+ 2019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.511288 (+ 953us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.513331 (+ 2043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.513984 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.514152 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:50.517728 (+ 3576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:50.517732 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:50.517736 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:50.517765 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:50.521648 (+ 3883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:50.522342 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:50.522348 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:50.522352 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:50.522487 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:50.522929 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:50.522934 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:50.522937 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:50.523251 (+ 314us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:50.523524 (+ 273us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:50.523826 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 12:20:50.524037 (+ 211us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":711,"thread_start_us":616,"threads_started":1}
I20260809 12:20:50.525303 11636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:50.504105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:38299 (local address 127.19.183.254:39553)
0809 12:20:50.504295 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:50.504300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:50.505367 (+ 1067us) server_negotiation.cc:407] Connection header received
0809 12:20:50.507339 (+ 1972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:50.507345 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:50.507457 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:50.507569 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:50.509821 (+ 2252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.510848 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.513074 (+ 2226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.513794 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.513923 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:50.518171 (+ 4248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:50.518179 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:50.518183 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:50.518241 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:50.521922 (+ 3681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:50.522604 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:50.522609 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:50.522613 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:50.522796 (+ 183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:50.523268 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:50.523273 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:50.523276 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:50.523529 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:50.523682 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:50.524473 (+ 791us) server_negotiation.cc:299] Negotiation successful
0809 12:20:50.524974 (+ 501us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 12:20:50.526057 12032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:50.504537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.195:38299)
0809 12:20:50.505213 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:50.505249 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:50.507023 (+ 1774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:50.507989 (+ 966us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:50.508000 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:50.508787 (+ 787us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:20:50.509638 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:50.509666 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.511023 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.511028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:50.512924 (+ 1896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:50.512940 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.514016 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.514021 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:50.514378 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:50.515298 (+ 920us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:20:50.515327 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:20:50.517976 (+ 2649us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:20:50.522049 (+ 4073us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:50.522057 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:50.522071 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:50.522470 (+ 399us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:50.522908 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:20:50.522913 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:20:50.522916 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:20:50.523109 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:20:50.523680 (+ 571us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:20:50.523688 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:20:50.523984 (+ 296us) client_negotiation.cc:769] Sending connection context
0809 12:20:50.524257 (+ 273us) client_negotiation.cc:240] Negotiation successful
0809 12:20:50.524759 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450,"thread_start_us":266,"threads_started":1}
I20260809 12:20:50.527311 12026 heartbeater.cc:347] Connected to a master server at 127.19.183.254:39553
I20260809 12:20:50.527621 12026 heartbeater.cc:464] Registering TS with master...
I20260809 12:20:50.528293 12026 heartbeater.cc:511] Master 127.19.183.254:39553 requested a full tablet report, sending...
I20260809 12:20:50.529659 11336 ts_manager.cc:194] Registered new tserver with Master: 2006b09388984ccbadee161a62f02309 (127.19.183.195:41153)
I20260809 12:20:50.530453 11336 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:38299
I20260809 12:20:50.539078 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:20:50.545065 11636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:50.540059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44882 (local address 127.19.183.254:39553)
0809 12:20:50.540220 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:50.540224 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:50.540581 (+ 357us) server_negotiation.cc:407] Connection header received
0809 12:20:50.540747 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:50.540751 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:50.540863 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:50.540988 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:20:50.541369 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.542060 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.543569 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.543951 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.544040 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:50.544493 (+ 453us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:20:50.544679 (+ 186us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 12:20:50.544916 (+ 237us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44882: token expired
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 12:20:50.545205 11636 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44882: token expired
W20260809 12:20:50.545831 11220 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.19.183.254:39553)
I20260809 12:20:50.558828 11636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:50.546427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44888 (local address 127.19.183.254:39553)
0809 12:20:50.546590 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:50.546595 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:50.546613 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:20:50.546746 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:50.546751 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:50.546809 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:50.546900 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:50.548274 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.548970 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.550370 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.550834 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.550963 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:50.553671 (+ 2708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:50.553677 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:50.553682 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:50.553721 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:50.556250 (+ 2529us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:50.556915 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:50.556921 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:50.556925 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:50.557046 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:50.557426 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:50.557431 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:50.557434 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:50.557689 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:50.557933 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:50.558251 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 12:20:50.558460 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 12:20:50.562047 11336 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44888:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:20:50.569195 11336 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:20:50.569300 11632 heartbeater.cc:503] Master 127.19.183.254:39553 was elected leader, sending a full tablet report...
I20260809 12:20:50.591109 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:50.580339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:40951 (local address 127.0.0.1:52712)
0809 12:20:50.581304 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:50.581364 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:50.581560 (+ 196us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:50.582942 (+ 1382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:50.582949 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:50.582982 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:20:50.583463 (+ 481us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:50.583478 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.585638 (+ 2160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.585643 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:50.588085 (+ 2442us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:50.588100 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.590119 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.590126 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:50.590357 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:50.590413 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 12:20:50.590518 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 12:20:50.590742 (+ 224us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":669,"thread_start_us":511,"threads_started":1}
I20260809 12:20:50.592849 12044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:50.580502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52712 (local address 127.19.183.194:40951)
0809 12:20:50.582181 (+ 1679us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:50.582190 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:50.582217 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:20:50.582315 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:50.582322 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:50.582611 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:50.583705 (+ 1094us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:20:50.583928 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.585470 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.588292 (+ 2822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.589844 (+ 1552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.592120 (+ 2276us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:50.592311 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:50.592385 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 12:20:50.592546 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1534,"thread_start_us":1443,"threads_started":1}
I20260809 12:20:50.592852 12045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:50.580466 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:41153 (local address 127.0.0.1:45874)
0809 12:20:50.582007 (+ 1541us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:50.582023 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:50.582140 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:50.584005 (+ 1865us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:50.584011 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:50.584039 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:20:50.584410 (+ 371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:50.584422 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.586792 (+ 2370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.586800 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:50.588819 (+ 2019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:50.588833 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.592094 (+ 3261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.592101 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:50.592272 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:50.592303 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 12:20:50.592399 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 12:20:50.592521 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1365,"thread_start_us":1243,"threads_started":1}
I20260809 12:20:50.594211 12046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:50.581030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45874 (local address 127.19.183.195:41153)
0809 12:20:50.582952 (+ 1922us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:50.582960 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:50.582990 (+ 30us) server_negotiation.cc:407] Connection header received
0809 12:20:50.583085 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:50.583095 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:50.583316 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:50.583713 (+ 397us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:20:50.584604 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.586389 (+ 1785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.589005 (+ 2616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.591544 (+ 2539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.591708 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:50.591879 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:50.593071 (+ 1192us) server_negotiation.cc:299] Negotiation successful
0809 12:20:50.593243 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1745,"thread_start_us":1559,"threads_started":1}
I20260809 12:20:50.596762 11735 tablet_service.cc:1467] Processing CreateTablet for tablet eaec7fd228ef4a73941993234c3ee740 (DEFAULT_TABLE table=test-table [id=afdf65b30e15480a897ddedda2cc37de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:20:50.598366 11735 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaec7fd228ef4a73941993234c3ee740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:20:50.598505 11927 tablet_service.cc:1467] Processing CreateTablet for tablet eaec7fd228ef4a73941993234c3ee740 (DEFAULT_TABLE table=test-table [id=afdf65b30e15480a897ddedda2cc37de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:20:50.599890 11927 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaec7fd228ef4a73941993234c3ee740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:20:50.601866 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:50.582090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43617 (local address 127.0.0.1:56234)
0809 12:20:50.591363 (+ 9273us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:50.591381 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:50.591476 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:50.595473 (+ 3997us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:50.595479 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:50.595505 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:20:50.595698 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:50.595713 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.597544 (+ 1831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.597551 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:50.599290 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:50.599306 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.601129 (+ 1823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.601136 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:50.601311 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:50.601347 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 12:20:50.601443 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 12:20:50.601588 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":9187,"thread_start_us":11158,"threads_started":1}
I20260809 12:20:50.602416 12051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:50.582507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56234 (local address 127.19.183.193:43617)
0809 12:20:50.594050 (+ 11543us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:50.594058 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:50.594086 (+ 28us) server_negotiation.cc:407] Connection header received
0809 12:20:50.594191 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:50.594198 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:50.595269 (+ 1071us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:50.595462 (+ 193us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:20:50.595901 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.597393 (+ 1492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.599524 (+ 2131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.600943 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.601147 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:50.601309 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:50.601919 (+ 610us) server_negotiation.cc:299] Negotiation successful
0809 12:20:50.602084 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11364,"thread_start_us":11226,"threads_started":1}
I20260809 12:20:50.605865 11532 tablet_service.cc:1467] Processing CreateTablet for tablet eaec7fd228ef4a73941993234c3ee740 (DEFAULT_TABLE table=test-table [id=afdf65b30e15480a897ddedda2cc37de]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:20:50.607213 11532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaec7fd228ef4a73941993234c3ee740. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:20:50.613188 12052 tablet_bootstrap.cc:492] T eaec7fd228ef4a73941993234c3ee740 P 2006b09388984ccbadee161a62f02309: Bootstrap starting.
I20260809 12:20:50.615010 12055 tablet_bootstrap.cc:492] T eaec7fd228ef4a73941993234c3ee740 P 45407bd46e4e4e4c94de4e814f066ede: Bootstrap starting.
I20260809 12:20:50.618258 12052 tablet_bootstrap.cc:654] T eaec7fd228ef4a73941993234c3ee740 P 2006b09388984ccbadee161a62f02309: Neither blocks nor log segments found. Creating new log.
I20260809 12:20:50.618517 12055 tablet_bootstrap.cc:654] T eaec7fd228ef4a73941993234c3ee740 P 45407bd46e4e4e4c94de4e814f066ede: Neither blocks nor log segments found. Creating new log.
I20260809 12:20:50.619699 12055 log.cc:824] T eaec7fd228ef4a73941993234c3ee740 P 45407bd46e4e4e4c94de4e814f066ede: Log is configured to *not* fsync() on all Append() calls
I20260809 12:20:50.619699 12052 log.cc:824] T eaec7fd228ef4a73941993234c3ee740 P 2006b09388984ccbadee161a62f02309: Log is configured to *not* fsync() on all Append() calls
I20260809 12:20:50.622473 12058 tablet_bootstrap.cc:492] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a: Bootstrap starting.
I20260809 12:20:50.623116 12055 tablet_bootstrap.cc:492] T eaec7fd228ef4a73941993234c3ee740 P 45407bd46e4e4e4c94de4e814f066ede: No bootstrap required, opened a new log
I20260809 12:20:50.623394 12055 ts_tablet_manager.cc:1397] T eaec7fd228ef4a73941993234c3ee740 P 45407bd46e4e4e4c94de4e814f066ede: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.007s
I20260809 12:20:50.623971 12052 tablet_bootstrap.cc:492] T eaec7fd228ef4a73941993234c3ee740 P 2006b09388984ccbadee161a62f02309: No bootstrap required, opened a new log
I20260809 12:20:50.624274 12052 ts_tablet_manager.cc:1397] T eaec7fd228ef4a73941993234c3ee740 P 2006b09388984ccbadee161a62f02309: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.006s
I20260809 12:20:50.626222 12058 tablet_bootstrap.cc:654] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a: Neither blocks nor log segments found. Creating new log.
I20260809 12:20:50.627390 12058 log.cc:824] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a: Log is configured to *not* fsync() on all Append() calls
I20260809 12:20:50.630228 12058 tablet_bootstrap.cc:492] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a: No bootstrap required, opened a new log
I20260809 12:20:50.630507 12058 ts_tablet_manager.cc:1397] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 12:20:50.631549 12052 raft_consensus.cc:348] T eaec7fd228ef4a73941993234c3ee740 P 2006b09388984ccbadee161a62f02309 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } } peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } }
I20260809 12:20:50.631892 12052 raft_consensus.cc:374] T eaec7fd228ef4a73941993234c3ee740 P 2006b09388984ccbadee161a62f02309 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:20:50.632104 12052 raft_consensus.cc:729] T eaec7fd228ef4a73941993234c3ee740 P 2006b09388984ccbadee161a62f02309 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2006b09388984ccbadee161a62f02309, State: Initialized, Role: FOLLOWER
I20260809 12:20:50.632751 12052 consensus_queue.cc:260] T eaec7fd228ef4a73941993234c3ee740 P 2006b09388984ccbadee161a62f02309 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } } peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } }
I20260809 12:20:50.633471 12055 raft_consensus.cc:348] T eaec7fd228ef4a73941993234c3ee740 P 45407bd46e4e4e4c94de4e814f066ede [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } } peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } }
I20260809 12:20:50.633816 12026 heartbeater.cc:503] Master 127.19.183.254:39553 was elected leader, sending a full tablet report...
I20260809 12:20:50.633880 12055 raft_consensus.cc:374] T eaec7fd228ef4a73941993234c3ee740 P 45407bd46e4e4e4c94de4e814f066ede [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:20:50.633983 12055 raft_consensus.cc:729] T eaec7fd228ef4a73941993234c3ee740 P 45407bd46e4e4e4c94de4e814f066ede [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45407bd46e4e4e4c94de4e814f066ede, State: Initialized, Role: FOLLOWER
I20260809 12:20:50.634462 12052 ts_tablet_manager.cc:1428] T eaec7fd228ef4a73941993234c3ee740 P 2006b09388984ccbadee161a62f02309: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260809 12:20:50.634613 12055 consensus_queue.cc:260] T eaec7fd228ef4a73941993234c3ee740 P 45407bd46e4e4e4c94de4e814f066ede [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } } peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } }
I20260809 12:20:50.635780 11821 heartbeater.cc:503] Master 127.19.183.254:39553 was elected leader, sending a full tablet report...
I20260809 12:20:50.636413 12055 ts_tablet_manager.cc:1428] T eaec7fd228ef4a73941993234c3ee740 P 45407bd46e4e4e4c94de4e814f066ede: Time spent starting tablet: real 0.013s user 0.007s sys 0.006s
I20260809 12:20:50.641844 12058 raft_consensus.cc:348] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } } peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } }
I20260809 12:20:50.642169 12058 raft_consensus.cc:374] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:20:50.642298 12058 raft_consensus.cc:729] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1469d62f6084bec9ac860cc7d88741a, State: Initialized, Role: FOLLOWER
I20260809 12:20:50.642900 12058 consensus_queue.cc:260] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } } peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } }
I20260809 12:20:50.644454 12058 ts_tablet_manager.cc:1428] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a: Time spent starting tablet: real 0.014s user 0.007s sys 0.006s
W20260809 12:20:50.753434 12029 tablet.cc:2369] T eaec7fd228ef4a73941993234c3ee740 P 2006b09388984ccbadee161a62f02309: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:20:50.777292 11824 tablet.cc:2369] T eaec7fd228ef4a73941993234c3ee740 P 45407bd46e4e4e4c94de4e814f066ede: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:20:50.786012 11634 tablet.cc:2369] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:20:50.804865 12064 raft_consensus.cc:482] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:20:50.805181 12064 raft_consensus.cc:504] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } } peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } }
I20260809 12:20:50.806897 12064 leader_election.cc:288] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45407bd46e4e4e4c94de4e814f066ede (127.19.183.194:40951), 2006b09388984ccbadee161a62f02309 (127.19.183.195:41153)
I20260809 12:20:50.814168 12044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:50.807353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:49571 (local address 127.19.183.194:40951)
0809 12:20:50.807554 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:50.807560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:50.808576 (+ 1016us) server_negotiation.cc:407] Connection header received
0809 12:20:50.808796 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:50.808800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:50.808860 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:50.808961 (+ 101us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:20:50.809633 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.810359 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.812337 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.813231 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.813409 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:50.813487 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:50.813771 (+ 284us) server_negotiation.cc:299] Negotiation successful
0809 12:20:50.813897 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 12:20:50.814217 12046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:50.807353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:41043 (local address 127.19.183.195:41153)
0809 12:20:50.807554 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:50.807560 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:50.808522 (+ 962us) server_negotiation.cc:407] Connection header received
0809 12:20:50.808742 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:50.808751 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:50.808812 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:50.808910 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:20:50.809565 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.810290 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.812373 (+ 2083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.813242 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.813460 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:50.813543 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:50.813840 (+ 297us) server_negotiation.cc:299] Negotiation successful
0809 12:20:50.813967 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 12:20:50.814316 12072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:50.807441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:40951 (local address 127.19.183.193:49571)
0809 12:20:50.808455 (+ 1014us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:50.808471 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:50.808634 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:50.808978 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:50.808982 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:50.809005 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:20:50.809448 (+ 443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:50.809457 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.810493 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.810498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:50.812183 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:50.812196 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.813443 (+ 1247us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.813450 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:50.813610 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:50.813635 (+ 25us) client_negotiation.cc:769] Sending connection context
0809 12:20:50.813743 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 12:20:50.813867 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":898,"mutex_wait_us":42,"thread_start_us":546,"threads_started":1}
I20260809 12:20:50.814760 12068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:50.807397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:41153 (local address 127.19.183.193:41043)
0809 12:20:50.808396 (+ 999us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:50.808417 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:20:50.808596 (+ 179us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:20:50.808935 (+ 339us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:20:50.808939 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:20:50.808966 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:20:50.809411 (+ 445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:50.809424 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.810450 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.810456 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:50.812188 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:20:50.812201 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.813529 (+ 1328us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.813535 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:20:50.813684 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:50.813706 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 12:20:50.813813 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 12:20:50.813933 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":853,"mutex_wait_us":97,"spinlock_wait_cycles":26368,"thread_start_us":663,"threads_started":1}
I20260809 12:20:50.815336 11758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaec7fd228ef4a73941993234c3ee740" candidate_uuid: "a1469d62f6084bec9ac860cc7d88741a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45407bd46e4e4e4c94de4e814f066ede" is_pre_election: true
I20260809 12:20:50.815653 11955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaec7fd228ef4a73941993234c3ee740" candidate_uuid: "a1469d62f6084bec9ac860cc7d88741a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2006b09388984ccbadee161a62f02309" is_pre_election: true
I20260809 12:20:50.815836 11758 raft_consensus.cc:2454] T eaec7fd228ef4a73941993234c3ee740 P 45407bd46e4e4e4c94de4e814f066ede [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1469d62f6084bec9ac860cc7d88741a in term 0.
I20260809 12:20:50.816152 11955 raft_consensus.cc:2454] T eaec7fd228ef4a73941993234c3ee740 P 2006b09388984ccbadee161a62f02309 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a1469d62f6084bec9ac860cc7d88741a in term 0.
I20260809 12:20:50.816534 11454 leader_election.cc:302] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45407bd46e4e4e4c94de4e814f066ede, a1469d62f6084bec9ac860cc7d88741a; no voters:
I20260809 12:20:50.816937 12064 raft_consensus.cc:2781] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:20:50.817063 12064 raft_consensus.cc:482] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:20:50.817195 12064 raft_consensus.cc:3037] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:20:50.819723 12064 raft_consensus.cc:504] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } } peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } }
I20260809 12:20:50.820452 12064 leader_election.cc:288] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [CANDIDATE]: Term 1 election: Requested vote from peers 45407bd46e4e4e4c94de4e814f066ede (127.19.183.194:40951), 2006b09388984ccbadee161a62f02309 (127.19.183.195:41153)
I20260809 12:20:50.820921 11758 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaec7fd228ef4a73941993234c3ee740" candidate_uuid: "a1469d62f6084bec9ac860cc7d88741a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45407bd46e4e4e4c94de4e814f066ede"
I20260809 12:20:50.820969 11955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaec7fd228ef4a73941993234c3ee740" candidate_uuid: "a1469d62f6084bec9ac860cc7d88741a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2006b09388984ccbadee161a62f02309"
I20260809 12:20:50.821159 11955 raft_consensus.cc:3037] T eaec7fd228ef4a73941993234c3ee740 P 2006b09388984ccbadee161a62f02309 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:20:50.821146 11758 raft_consensus.cc:3037] T eaec7fd228ef4a73941993234c3ee740 P 45407bd46e4e4e4c94de4e814f066ede [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:20:50.823585 11758 raft_consensus.cc:2454] T eaec7fd228ef4a73941993234c3ee740 P 45407bd46e4e4e4c94de4e814f066ede [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1469d62f6084bec9ac860cc7d88741a in term 1.
I20260809 12:20:50.823611 11955 raft_consensus.cc:2454] T eaec7fd228ef4a73941993234c3ee740 P 2006b09388984ccbadee161a62f02309 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a1469d62f6084bec9ac860cc7d88741a in term 1.
I20260809 12:20:50.824028 11454 leader_election.cc:302] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45407bd46e4e4e4c94de4e814f066ede, a1469d62f6084bec9ac860cc7d88741a; no voters:
I20260809 12:20:50.824333 12064 raft_consensus.cc:2781] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:20:50.824820 12064 raft_consensus.cc:686] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 1 LEADER]: Becoming Leader. State: Replica: a1469d62f6084bec9ac860cc7d88741a, State: Running, Role: LEADER
I20260809 12:20:50.825331 12064 consensus_queue.cc:237] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } } peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } }
I20260809 12:20:50.830120 11332 catalog_manager.cc:5495] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a reported cstate change: term changed from 0 to 1, leader changed from <none> to a1469d62f6084bec9ac860cc7d88741a (127.19.183.193). New cstate: current_term: 1 leader_uuid: "a1469d62f6084bec9ac860cc7d88741a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } health_report { overall_health: HEALTHY } } }
I20260809 12:20:50.889360 12051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:50.883761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56236 (local address 127.19.183.193:43617)
0809 12:20:50.883965 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:50.883973 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:50.884004 (+ 31us) server_negotiation.cc:407] Connection header received
0809 12:20:50.884267 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:50.884271 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:50.884360 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:50.884524 (+ 164us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:20:50.884907 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.885660 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.887204 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.887627 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.887769 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:50.888255 (+ 486us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:20:50.888726 (+ 471us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:20:50.888883 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:50.888962 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 12:20:50.889040 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 12:20:50.897377 11955 raft_consensus.cc:1261] T eaec7fd228ef4a73941993234c3ee740 P 2006b09388984ccbadee161a62f02309 [term 1 FOLLOWER]: Refusing update from remote peer a1469d62f6084bec9ac860cc7d88741a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:20:50.897375 11758 raft_consensus.cc:1261] T eaec7fd228ef4a73941993234c3ee740 P 45407bd46e4e4e4c94de4e814f066ede [term 1 FOLLOWER]: Refusing update from remote peer a1469d62f6084bec9ac860cc7d88741a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:20:50.898397 12064 consensus_queue.cc:1035] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [LEADER]: Connected to new peer: Peer: permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:20:50.898674 12074 consensus_queue.cc:1035] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:20:50.910995 12083 mvcc.cc:205] Tried to move back new op lower bound from 7316594896465616896 to 7316594896184487936. Current Snapshot: MvccSnapshot[applied={T|T < 7316594896465616896}]
I20260809 12:20:50.912365 12080 mvcc.cc:205] Tried to move back new op lower bound from 7316594896465616896 to 7316594896184487936. Current Snapshot: MvccSnapshot[applied={T|T < 7316594896465616896}]
I20260809 12:20:50.913278 12079 mvcc.cc:205] Tried to move back new op lower bound from 7316594896465616896 to 7316594896184487936. Current Snapshot: MvccSnapshot[applied={T|T < 7316594896465616896 or (T in {7316594896465616896})}]
I20260809 12:20:50.924659 11336 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:20:50.924952 11336 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44888:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:20:50.928476 11336 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae: Sending DeleteTablet for 3 replicas of tablet eaec7fd228ef4a73941993234c3ee740
I20260809 12:20:50.929626 11735 tablet_service.cc:1514] Processing DeleteTablet for tablet eaec7fd228ef4a73941993234c3ee740 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:20:50 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:52712
I20260809 12:20:50.929844 11532 tablet_service.cc:1514] Processing DeleteTablet for tablet eaec7fd228ef4a73941993234c3ee740 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:20:50 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:56234
I20260809 12:20:50.929930 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 11393
I20260809 12:20:50.929845 11927 tablet_service.cc:1514] Processing DeleteTablet for tablet eaec7fd228ef4a73941993234c3ee740 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:20:50 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:45874
I20260809 12:20:50.942178 12098 tablet_replica.cc:330] T eaec7fd228ef4a73941993234c3ee740 P 2006b09388984ccbadee161a62f02309: stopping tablet replica
I20260809 12:20:50.942646 12098 raft_consensus.cc:2229] T eaec7fd228ef4a73941993234c3ee740 P 2006b09388984ccbadee161a62f02309 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:20:50.943266 12098 raft_consensus.cc:2258] T eaec7fd228ef4a73941993234c3ee740 P 2006b09388984ccbadee161a62f02309 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:20:50.944213 12095 tablet_replica.cc:330] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a: stopping tablet replica
I20260809 12:20:50.944510 12093 tablet_replica.cc:330] T eaec7fd228ef4a73941993234c3ee740 P 45407bd46e4e4e4c94de4e814f066ede: stopping tablet replica
I20260809 12:20:50.944631 12095 raft_consensus.cc:2229] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:20:50.944921 12093 raft_consensus.cc:2229] T eaec7fd228ef4a73941993234c3ee740 P 45407bd46e4e4e4c94de4e814f066ede [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:20:50.945328 12095 raft_consensus.cc:2258] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:20:50.945385 12093 raft_consensus.cc:2258] T eaec7fd228ef4a73941993234c3ee740 P 45407bd46e4e4e4c94de4e814f066ede [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:20:50.947446 12093 ts_tablet_manager.cc:1905] T eaec7fd228ef4a73941993234c3ee740 P 45407bd46e4e4e4c94de4e814f066ede: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:20:50.947568 12098 ts_tablet_manager.cc:1905] T eaec7fd228ef4a73941993234c3ee740 P 2006b09388984ccbadee161a62f02309: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:20:50.954118 12051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:50.930835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56248 (local address 127.19.183.193:43617)
0809 12:20:50.931060 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:50.931068 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:50.931698 (+ 630us) server_negotiation.cc:407] Connection header received
0809 12:20:50.931777 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:50.931791 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:50.931891 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:50.932014 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:50.934520 (+ 2506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.936194 (+ 1674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.938864 (+ 2670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:50.939893 (+ 1029us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:50.940065 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:50.945107 (+ 5042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:50.945122 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:50.945139 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:50.945196 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:50.949827 (+ 4631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:50.950768 (+ 941us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:50.950776 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:50.950780 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:50.950952 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:50.951456 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:50.951461 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:50.951464 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:50.951965 (+ 501us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:50.952267 (+ 302us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:50.953501 (+ 1234us) server_negotiation.cc:299] Negotiation successful
0809 12:20:50.953756 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260809 12:20:50.954645 12098 ts_tablet_manager.cc:1918] T eaec7fd228ef4a73941993234c3ee740 P 2006b09388984ccbadee161a62f02309: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:20:50.954695 12093 ts_tablet_manager.cc:1918] T eaec7fd228ef4a73941993234c3ee740 P 45407bd46e4e4e4c94de4e814f066ede: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:20:50.954835 12098 log.cc:1196] T eaec7fd228ef4a73941993234c3ee740 P 2006b09388984ccbadee161a62f02309: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/eaec7fd228ef4a73941993234c3ee740
I20260809 12:20:50.954900 12093 log.cc:1196] T eaec7fd228ef4a73941993234c3ee740 P 45407bd46e4e4e4c94de4e814f066ede: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/eaec7fd228ef4a73941993234c3ee740
I20260809 12:20:50.955081 11626 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:20:50.955408 12098 ts_tablet_manager.cc:1939] T eaec7fd228ef4a73941993234c3ee740 P 2006b09388984ccbadee161a62f02309: Deleting consensus metadata
I20260809 12:20:50.955469 12093 ts_tablet_manager.cc:1939] T eaec7fd228ef4a73941993234c3ee740 P 45407bd46e4e4e4c94de4e814f066ede: Deleting consensus metadata
I20260809 12:20:50.957620 11315 catalog_manager.cc:4841] TS 45407bd46e4e4e4c94de4e814f066ede (127.19.183.194:40951): tablet eaec7fd228ef4a73941993234c3ee740 (table test-table [id=afdf65b30e15480a897ddedda2cc37de]) successfully deleted
I20260809 12:20:50.957739 11314 catalog_manager.cc:4841] TS 2006b09388984ccbadee161a62f02309 (127.19.183.195:41153): tablet eaec7fd228ef4a73941993234c3ee740 (table test-table [id=afdf65b30e15480a897ddedda2cc37de]) successfully deleted
I20260809 12:20:51.053464 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 11393
W20260809 12:20:51.079866 11314 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.183.193:43617 (error 108)
I20260809 12:20:51.080071 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 11642
I20260809 12:20:51.081835 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:51.081341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43617 (local address 127.0.0.1:56262)
0809 12:20:51.081519 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:51.081647 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 12:20:51.082096 11314 catalog_manager.cc:4584] TS a1469d62f6084bec9ac860cc7d88741a (127.19.183.193:43617): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eaec7fd228ef4a73941993234c3ee740: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43617: connect: Connection refused (error 111)
I20260809 12:20:51.094019 12044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:51.081069 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52726 (local address 127.19.183.194:40951)
0809 12:20:51.081275 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:51.081281 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:51.081303 (+ 22us) server_negotiation.cc:407] Connection header received
0809 12:20:51.081497 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:51.081503 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:51.081595 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:51.081682 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:51.083272 (+ 1590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:51.084016 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:51.085349 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:51.085745 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:51.085833 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:51.088546 (+ 2713us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:51.088559 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:51.088573 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:51.088608 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:51.091574 (+ 2966us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:51.092254 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:51.092261 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:51.092263 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:51.092373 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:51.092690 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:51.092693 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:51.092695 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:51.093021 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:51.093365 (+ 344us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:51.093544 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 12:20:51.093697 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 12:20:51.094691 11815 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:20:51.135802 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:51.135363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43617 (local address 127.0.0.1:56266)
0809 12:20:51.135563 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:51.135669 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 12:20:51.136883 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:51.136505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43617 (local address 127.0.0.1:56270)
0809 12:20:51.136677 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:51.136744 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
I20260809 12:20:51.183422 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 11642
I20260809 12:20:51.197574 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:51.196987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43617 (local address 127.0.0.1:56282)
0809 12:20:51.197255 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:51.197377 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 12:20:51.198820 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:51.198294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43617 (local address 127.0.0.1:56292)
0809 12:20:51.198481 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:51.198601 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260809 12:20:51.208961 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 11827
I20260809 12:20:51.222364 12046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:51.209725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45886 (local address 127.19.183.195:41153)
0809 12:20:51.209906 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 12:20:51.209911 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:20:51.209930 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:20:51.210051 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:20:51.210055 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:20:51.210132 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:20:51.210215 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:20:51.211705 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:51.212494 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:51.213755 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:20:51.214136 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:20:51.214223 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:20:51.216846 (+ 2623us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:20:51.216861 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:20:51.216877 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:20:51.216908 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:20:51.219532 (+ 2624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:51.220159 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:51.220167 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:51.220169 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:51.220307 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:20:51.220728 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:20:51.220734 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:20:51.220740 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:20:51.221214 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:20:51.221495 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0809 12:20:51.221732 (+ 237us) server_negotiation.cc:299] Negotiation successful
0809 12:20:51.221984 (+ 252us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 12:20:51.223069 12017 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:20:51.279358 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:51.278869 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43617 (local address 127.0.0.1:56298)
0809 12:20:51.279071 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:51.279210 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260809 12:20:51.280424 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:51.279991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43617 (local address 127.0.0.1:56302)
0809 12:20:51.280197 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:51.280279 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260809 12:20:51.311172 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 11827
I20260809 12:20:51.338395 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 11287
I20260809 12:20:51.339709 11377 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:20:51.453801 12043 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:20:51.452969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:43617 (local address 127.0.0.1:34796)
0809 12:20:51.453280 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0809 12:20:51.453577 (+ 297us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:43617: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260809 12:20:51.453977 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 11287
I20260809 12:21:11.483117 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:39553
--webserver_interface=127.19.183.254
--webserver_port=43151
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:39553
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:21:11.830437 12110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:11.830713 12110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:11.830881 12110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:11.837908 12110 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 12:21:11.837996 12110 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 12:21:11.838052 12110 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 12:21:11.838104 12110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:21:11.838151 12110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:11.838220 12110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:11.838259 12110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:21:11.838321 12110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:21:11.847718 12110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:39553
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:39553
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=43151
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:11.848897 12110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:11.850157 12110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:11.861930 12116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:11.862294 12123 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:11.862433 12118 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:11.862994 12110 server_base.cc:1034] running on GCE node
I20260809 12:21:11.863749 12110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:11.865051 12110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:11.866302 12110 hybrid_clock.cc:648] HybridClock initialized: now 1786278071866254 us; error 45 us; skew 500 ppm
I20260809 12:21:11.866621 12110 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:21:11 dist-test-slave-39l8 krb5kdc[10458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278071, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:21:11.873560 12110 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:21:11.875165 12110 webserver.cc:460] Webserver started at http://127.19.183.254:43151/ using document root <none> and password file <none>
I20260809 12:21:11.875839 12110 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:11.875933 12110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:11.881307 12110 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 12:21:11.884078 12132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:11.885437 12110 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 12:21:11.885623 12110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "a4372730a7844930bf3bfbe08c4d4bae"
format_stamp: "Formatted at 2026-08-09 12:20:25 on dist-test-slave-39l8"
server_key: "08397aa2f3108341449c78ae979514b6"
server_key_iv: "52e6701d155b94d0388b3eb1541940b3"
server_key_version: "encryptionkey@0"
I20260809 12:21:11.886618 12110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:11.905624 12110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:11.909439 12110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:11.909746 12110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:11.928747 12110 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:39553
I20260809 12:21:11.928781 12210 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:39553 every 8 connection(s)
I20260809 12:21:11.930243 12110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:21:11.935217 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 12110
I20260809 12:21:11.936124 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:43617
--local_ip_for_outbound_sockets=127.19.183.193
--tserver_master_addrs=127.19.183.254:39553
--webserver_port=39839
--webserver_interface=127.19.183.193
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:21:11.937307 12212 sys_catalog.cc:263] Verifying existing consensus state
I20260809 12:21:11.940512 12212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae: Bootstrap starting.
I20260809 12:21:11.960898 12212 log.cc:824] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:11.977089 12212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:21:11.977684 12212 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae: Bootstrap complete.
I20260809 12:21:11.986977 12212 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4372730a7844930bf3bfbe08c4d4bae" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39553 } }
I20260809 12:21:11.987876 12212 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4372730a7844930bf3bfbe08c4d4bae, State: Initialized, Role: FOLLOWER
I20260809 12:21:11.988452 12212 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4372730a7844930bf3bfbe08c4d4bae" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39553 } }
I20260809 12:21:11.988729 12212 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:21:11.988865 12212 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:21:11.989053 12212 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 2 FOLLOWER]: Advancing to term 3
I20260809 12:21:11.993162 12212 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4372730a7844930bf3bfbe08c4d4bae" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39553 } }
I20260809 12:21:11.993634 12212 leader_election.cc:302] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4372730a7844930bf3bfbe08c4d4bae; no voters:
I20260809 12:21:11.994262 12212 leader_election.cc:288] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 12:21:11.994472 12217 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 3 FOLLOWER]: Leader election won for term 3
I20260809 12:21:11.995636 12217 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [term 3 LEADER]: Becoming Leader. State: Replica: a4372730a7844930bf3bfbe08c4d4bae, State: Running, Role: LEADER
I20260809 12:21:11.996279 12212 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:21:11.996347 12217 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4372730a7844930bf3bfbe08c4d4bae" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39553 } }
I20260809 12:21:11.999478 12218 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a4372730a7844930bf3bfbe08c4d4bae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4372730a7844930bf3bfbe08c4d4bae" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39553 } } }
I20260809 12:21:11.999989 12218 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [sys.catalog]: This master's current role is: LEADER
I20260809 12:21:11.999454 12220 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4372730a7844930bf3bfbe08c4d4bae. Latest consensus state: current_term: 3 leader_uuid: "a4372730a7844930bf3bfbe08c4d4bae" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4372730a7844930bf3bfbe08c4d4bae" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 39553 } } }
I20260809 12:21:12.000541 12220 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae [sys.catalog]: This master's current role is: LEADER
I20260809 12:21:12.003522 12225 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:21:12.012023 12225 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:21:12.013496 12225 catalog_manager.cc:1259] Loaded cluster ID: e120d0e86c994ebd97187a943dec94bc
I20260809 12:21:12.013645 12225 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:21:12.017688 12225 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:21:12.019430 12225 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae: Loaded TSK: 0
I20260809 12:21:12.019675 12225 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a4372730a7844930bf3bfbe08c4d4bae: Loaded TSK: 1
F20260809 12:21:12.020927 12225 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f49d22b99ad google::LogMessage::Fail() at ??:0
@ 0x7f49d22bc88a google::LogMessage::SendToLog() at ??:0
@ 0x7f49d22b94de google::LogMessage::Flush() at ??:0
@ 0x7f49d22baa19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f49d601d7d2 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f49d600c73e kudu::security::TokenSigner::ImportKeys() at ??:0
I20260809 12:21:12.044180 12248 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
@ 0x7f49e7ce81c0 kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f49e7ce7783 kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f49e7d732a7 kudu::master::CatalogManager::PrepareForLeadershipTask()::$_10::operator()() at ??:0
@ 0x7f49e7d730c4 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x7f49e7dbd5e9 std::function<>::operator()() at ??:0
@ 0x7f49e7cebcdd kudu::master::CatalogManager::PrepareForLeadershipTask()::$_6::operator()() at ??:0
@ 0x7f49e7ce9c36 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f49e7d71610 kudu::master::CatalogManager::ElectedAsLeaderCb()::$_2::operator()() at ??:0
@ 0x7f49e7d71439 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f49d4f05345 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f49d4f0891c kudu::ThreadPool::CreateThread()::$_2::operator()() at ??:0
@ 0x7f49d4f08799 std::_Function_handler<>::_M_invoke() at ??:0
@ 0x9edb7d std::function<>::operator()() at ??:?
@ 0x7f49d4ee3eaf kudu::Thread::SuperviseThread() at ??:0
@ 0x7f49d81bd6db start_thread at ??:0
@ 0x7f49d115361f clone at ??:0
W20260809 12:21:12.350184 12215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:12.350469 12215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:12.350646 12215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:12.357897 12215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:12.358039 12215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:12.358233 12215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:21:12.367887 12215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:43617
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=39839
--tserver_master_addrs=127.19.183.254:39553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:12.369180 12215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:12.370464 12215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:12.383713 12294 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:12.383852 12285 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:12.383845 12291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:12.384927 12215 server_base.cc:1034] running on GCE node
I20260809 12:21:12.385520 12215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:12.386435 12215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:12.387677 12215 hybrid_clock.cc:648] HybridClock initialized: now 1786278072387620 us; error 76 us; skew 500 ppm
I20260809 12:21:12.387984 12215 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:21:12 dist-test-slave-39l8 krb5kdc[10458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278072, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:21:12.391618 12215 init.cc:377] Logged in from keytab as kudu/127.19.183.193@KRBTEST.COM (short username kudu)
I20260809 12:21:12.393095 12215 webserver.cc:460] Webserver started at http://127.19.183.193:39839/ using document root <none> and password file <none>
I20260809 12:21:12.393811 12215 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:12.393962 12215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:12.399196 12215 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:21:12.402065 12301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:12.403394 12215 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:21:12.403576 12215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "a1469d62f6084bec9ac860cc7d88741a"
format_stamp: "Formatted at 2026-08-09 12:20:26 on dist-test-slave-39l8"
server_key: "9eeff0c547456e0f543319953a68fb21"
server_key_iv: "77b6387c37d8de70616925c8fd2b65cb"
server_key_version: "encryptionkey@0"
I20260809 12:21:12.404532 12215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:12.433794 12215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:12.437898 12215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:12.438202 12215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:12.439538 12215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:21:12.441470 12315 ts_tablet_manager.cc:542] Loading tablet metadata (0/1 complete)
I20260809 12:21:12.447016 12215 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 12:21:12.447175 12215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260809 12:21:12.447325 12215 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 12:21:12.450470 12215 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 12:21:12.450582 12215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.000s sys 0.000s
I20260809 12:21:12.450973 12315 tablet_bootstrap.cc:492] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a: Bootstrap starting.
I20260809 12:21:12.459000 12315 log.cc:824] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:12.472981 12315 tablet_bootstrap.cc:492] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 12:21:12.473680 12315 tablet_bootstrap.cc:492] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a: Bootstrap complete.
I20260809 12:21:12.474291 12315 ts_tablet_manager.cc:1397] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a: Time spent bootstrapping tablet: real 0.024s user 0.021s sys 0.001s
I20260809 12:21:12.485343 12315 raft_consensus.cc:348] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } } peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } }
I20260809 12:21:12.486526 12315 raft_consensus.cc:729] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a1469d62f6084bec9ac860cc7d88741a, State: Initialized, Role: FOLLOWER
I20260809 12:21:12.487396 12315 consensus_queue.cc:260] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } } peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } }
I20260809 12:21:12.489132 12315 ts_tablet_manager.cc:1428] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a: Time spent starting tablet: real 0.015s user 0.007s sys 0.007s
I20260809 12:21:12.497833 12215 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:43617
I20260809 12:21:12.497865 12477 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:43617 every 8 connection(s)
I20260809 12:21:12.499213 12215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:21:12.503891 12481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:12.501996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.193:47949)
0809 12:21:12.502833 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:12.503082 (+ 249us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":515,"thread_start_us":298,"threads_started":1}
W20260809 12:21:12.504485 12479 heartbeater.cc:644] Failed to heartbeat to 127.19.183.254:39553 (0 consecutive failures): Network error: Failed to ping master at 127.19.183.254:39553: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
I20260809 12:21:12.505306 12481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:12.504985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.193:60299)
0809 12:21:12.505141 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:12.505210 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 12:21:12.506220 12481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:12.505867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.193:53305)
0809 12:21:12.506017 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:12.506092 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 12:21:12.506477 12479 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 12:21:12.509696 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 12215
I20260809 12:21:12.510362 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:40951
--local_ip_for_outbound_sockets=127.19.183.194
--tserver_master_addrs=127.19.183.254:39553
--webserver_port=41643
--webserver_interface=127.19.183.194
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:21:12.859112 12482 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:12.859397 12482 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:12.859608 12482 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:12.866864 12482 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:12.867009 12482 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:12.867203 12482 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:21:12.877424 12482 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:40951
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=41643
--tserver_master_addrs=127.19.183.254:39553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:12.878831 12482 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:12.880187 12482 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:12.893400 12487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:12.893366 12488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:12.893321 12493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:12.894596 12482 server_base.cc:1034] running on GCE node
I20260809 12:21:12.895227 12482 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:12.896199 12482 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:12.897433 12482 hybrid_clock.cc:648] HybridClock initialized: now 1786278072897373 us; error 47 us; skew 500 ppm
I20260809 12:21:12.897763 12482 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:21:12 dist-test-slave-39l8 krb5kdc[10458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278072, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:21:12.900909 12482 init.cc:377] Logged in from keytab as kudu/127.19.183.194@KRBTEST.COM (short username kudu)
I20260809 12:21:12.902436 12482 webserver.cc:460] Webserver started at http://127.19.183.194:41643/ using document root <none> and password file <none>
I20260809 12:21:12.903190 12482 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:12.903340 12482 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:12.908759 12482 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.005s sys 0.000s
I20260809 12:21:12.911577 12503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:12.912911 12482 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:21:12.913117 12482 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "45407bd46e4e4e4c94de4e814f066ede"
format_stamp: "Formatted at 2026-08-09 12:20:26 on dist-test-slave-39l8"
server_key: "cb5f28f572af396b9f51f43fbb331c0f"
server_key_iv: "177c99be4d213c1119edbb48e64b02d8"
server_key_version: "encryptionkey@0"
I20260809 12:21:12.914130 12482 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:12.935969 12482 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:12.940044 12482 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:12.940341 12482 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:12.941375 12482 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:21:12.942885 12482 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:21:12.943019 12482 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:12.943158 12482 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:21:12.943259 12482 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:12.977900 12482 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:40951
I20260809 12:21:12.978006 12672 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:40951 every 8 connection(s)
I20260809 12:21:12.979463 12482 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:21:12.982883 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 12482
I20260809 12:21:12.983523 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:41153
--local_ip_for_outbound_sockets=127.19.183.195
--tserver_master_addrs=127.19.183.254:39553
--webserver_port=39099
--webserver_interface=127.19.183.195
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:21:12.984508 12678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:12.982407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.194:58091)
0809 12:21:12.983282 (+ 875us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:12.983615 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":383,"threads_started":1}
W20260809 12:21:12.985194 12674 heartbeater.cc:644] Failed to heartbeat to 127.19.183.254:39553 (0 consecutive failures): Network error: Failed to ping master at 127.19.183.254:39553: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
I20260809 12:21:12.986060 12678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:12.985750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.194:57511)
0809 12:21:12.985895 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:12.985963 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260809 12:21:12.986955 12678 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:12.986555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.194:58019)
0809 12:21:12.986700 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:12.986840 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 12:21:12.987228 12674 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 12:21:13.331503 12679 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:13.331758 12679 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:13.331960 12679 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:13.338795 12679 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:13.338887 12679 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:13.339087 12679 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:21:13.348444 12679 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:41153
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=39099
--tserver_master_addrs=127.19.183.254:39553
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:13.349678 12679 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:13.351042 12679 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:13.364254 12691 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:13.364368 12687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:13.364346 12694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:13.365258 12679 server_base.cc:1034] running on GCE node
I20260809 12:21:13.365805 12679 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:13.366621 12679 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:13.367815 12679 hybrid_clock.cc:648] HybridClock initialized: now 1786278073367757 us; error 47 us; skew 500 ppm
I20260809 12:21:13.368076 12679 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:21:13 dist-test-slave-39l8 krb5kdc[10458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278073, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:21:13.371114 12679 init.cc:377] Logged in from keytab as kudu/127.19.183.195@KRBTEST.COM (short username kudu)
I20260809 12:21:13.372510 12679 webserver.cc:460] Webserver started at http://127.19.183.195:39099/ using document root <none> and password file <none>
I20260809 12:21:13.373174 12679 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:13.373327 12679 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:13.378432 12679 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:21:13.381080 12703 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:13.382284 12679 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 12:21:13.382462 12679 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "2006b09388984ccbadee161a62f02309"
format_stamp: "Formatted at 2026-08-09 12:20:27 on dist-test-slave-39l8"
server_key: "d914c256b31aaeee56b5cbe8b162867c"
server_key_iv: "89076bb2c71a8027af48ebe7127dbb41"
server_key_version: "encryptionkey@0"
I20260809 12:21:13.383446 12679 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:13.407094 12679 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:13.410955 12679 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:13.411258 12679 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:13.412197 12679 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:21:13.413533 12679 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:21:13.413662 12679 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:13.413791 12679 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:21:13.413889 12679 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:13.448441 12679 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:41153
I20260809 12:21:13.448531 12870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:41153 every 8 connection(s)
I20260809 12:21:13.449919 12679 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:21:13.454887 12873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:13.452993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.195:52985)
0809 12:21:13.453730 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:13.453995 (+ 265us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":286,"threads_started":1}
W20260809 12:21:13.455593 12871 heartbeater.cc:644] Failed to heartbeat to 127.19.183.254:39553 (0 consecutive failures): Network error: Failed to ping master at 127.19.183.254:39553: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
I20260809 12:21:13.456405 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 12679
I20260809 12:21:13.456503 12873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:13.456166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.195:56203)
0809 12:21:13.456311 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:13.456376 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 12:21:13.457399 12873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:13.457085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.195:37825)
0809 12:21:13.457224 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:13.457296 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 12:21:13.457787 12871 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 12:21:13.507818 12875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:13.507008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.193:54275)
0809 12:21:13.507450 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:13.507560 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":220,"threads_started":1}
I20260809 12:21:13.792790 12877 raft_consensus.cc:482] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:21:13.793099 12877 raft_consensus.cc:504] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } } peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } }
I20260809 12:21:13.794842 12877 leader_election.cc:288] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45407bd46e4e4e4c94de4e814f066ede (127.19.183.194:40951), 2006b09388984ccbadee161a62f02309 (127.19.183.195:41153)
Aug 09 12:21:13 dist-test-slave-39l8 krb5kdc[10458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278072, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for kudu/127.19.183.195@KRBTEST.COM
Aug 09 12:21:13 dist-test-slave-39l8 krb5kdc[10458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278072, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for kudu/127.19.183.194@KRBTEST.COM
I20260809 12:21:13.818413 12880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:13.795396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:39347 (local address 127.19.183.194:40951)
0809 12:21:13.797695 (+ 2299us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:13.797705 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:13.797737 (+ 32us) server_negotiation.cc:407] Connection header received
0809 12:21:13.799213 (+ 1476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:13.799221 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:13.799576 (+ 355us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:13.799896 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:13.802077 (+ 2181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:13.803320 (+ 1243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:13.805120 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:13.805755 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:13.805955 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:13.810528 (+ 4573us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:13.810537 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:13.810551 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:13.810608 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:13.814491 (+ 3883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:13.815322 (+ 831us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:13.815330 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:13.815339 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:13.815536 (+ 197us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:13.815957 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:13.815962 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:13.815965 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:13.816532 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:13.816717 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:13.817757 (+ 1040us) server_negotiation.cc:299] Negotiation successful
0809 12:21:13.818108 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2157,"thread_start_us":2050,"threads_started":1}
I20260809 12:21:13.818529 12875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:13.795198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:40951 (local address 127.19.183.193:39347)
0809 12:21:13.795526 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:13.795570 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:13.796161 (+ 591us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:13.800059 (+ 3898us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:13.800070 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:13.800850 (+ 780us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:21:13.801872 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:13.801893 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:13.803543 (+ 1650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:13.803547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:13.804966 (+ 1419us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:13.804979 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:13.805998 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:13.806009 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:13.806321 (+ 312us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:13.807151 (+ 830us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:21:13.807174 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:21:13.810256 (+ 3082us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:21:13.814615 (+ 4359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:21:13.814629 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:21:13.814632 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:21:13.815182 (+ 550us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:21:13.815676 (+ 494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:21:13.815682 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:21:13.815685 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:21:13.815846 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:21:13.816689 (+ 843us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:21:13.816696 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:21:13.817153 (+ 457us) client_negotiation.cc:769] Sending connection context
0809 12:21:13.817481 (+ 328us) client_negotiation.cc:240] Negotiation successful
0809 12:21:13.818113 (+ 632us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178,"mutex_wait_us":397,"spinlock_wait_cycles":10112}
I20260809 12:21:13.819473 12882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:13.795784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:55889 (local address 127.19.183.195:41153)
0809 12:21:13.797553 (+ 1769us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:13.797568 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:13.797613 (+ 45us) server_negotiation.cc:407] Connection header received
0809 12:21:13.799215 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:13.799224 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:13.799578 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:13.799851 (+ 273us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:13.802073 (+ 2222us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:13.803341 (+ 1268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:13.805116 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:13.805755 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:13.805983 (+ 228us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:13.810212 (+ 4229us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:13.810220 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:13.810232 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:13.810267 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:13.814161 (+ 3894us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:13.815051 (+ 890us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:13.815057 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:13.815062 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:13.815307 (+ 245us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:13.815798 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:13.815802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:13.815804 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:13.816532 (+ 728us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:13.816717 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:13.818864 (+ 2147us) server_negotiation.cc:299] Negotiation successful
0809 12:21:13.819190 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1547,"thread_start_us":1356,"threads_started":1}
I20260809 12:21:13.819694 12881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:13.795404 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:41153 (local address 127.19.183.193:55889)
0809 12:21:13.797116 (+ 1712us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:13.797144 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:13.797257 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:13.800300 (+ 3043us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:13.800308 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:13.800945 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:21:13.801871 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:13.801894 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:13.803523 (+ 1629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:13.803541 (+ 18us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:13.804967 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:13.804980 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:13.806031 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:13.806040 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:13.806326 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:13.807196 (+ 870us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:21:13.807212 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:21:13.809922 (+ 2710us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:21:13.814315 (+ 4393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:21:13.814322 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:21:13.814332 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:21:13.814919 (+ 587us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:21:13.815432 (+ 513us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:21:13.815435 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:21:13.815438 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:21:13.815676 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:21:13.816689 (+ 1013us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:21:13.816696 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:21:13.817153 (+ 457us) client_negotiation.cc:769] Sending connection context
0809 12:21:13.817516 (+ 363us) client_negotiation.cc:240] Negotiation successful
0809 12:21:13.817998 (+ 482us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1545,"mutex_wait_us":184,"spinlock_wait_cycles":35968,"thread_start_us":1311,"threads_started":1}
W20260809 12:21:13.822175 12305 leader_election.cc:341] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 45407bd46e4e4e4c94de4e814f066ede (127.19.183.194:40951): Not found: Tablet not found: eaec7fd228ef4a73941993234c3ee740
W20260809 12:21:13.822573 12304 leader_election.cc:341] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2006b09388984ccbadee161a62f02309 (127.19.183.195:41153): Not found: Tablet not found: eaec7fd228ef4a73941993234c3ee740
I20260809 12:21:13.822683 12304 leader_election.cc:302] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1469d62f6084bec9ac860cc7d88741a; no voters: 2006b09388984ccbadee161a62f02309, 45407bd46e4e4e4c94de4e814f066ede
I20260809 12:21:13.823196 12877 raft_consensus.cc:2726] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 12:21:13.988703 12887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:13.987757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.194:50257)
0809 12:21:13.988325 (+ 568us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:13.988465 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":467,"thread_start_us":311,"threads_started":1}
I20260809 12:21:14.459213 12888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:14.458410 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.195:34403)
0809 12:21:14.458890 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:14.459031 (+ 141us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":382,"thread_start_us":221,"threads_started":1}
I20260809 12:21:14.509265 12890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:14.508634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.193:33427)
0809 12:21:14.509022 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:14.509111 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":193,"threads_started":1}
I20260809 12:21:14.990391 12891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:14.989577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.194:60245)
0809 12:21:14.990037 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:14.990164 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":262,"threads_started":1}
I20260809 12:21:15.460806 12892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:15.460081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.195:47005)
0809 12:21:15.460541 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:15.460633 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":243,"threads_started":1}
I20260809 12:21:15.510936 12894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:15.510105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.193:37451)
0809 12:21:15.510601 (+ 496us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:15.510685 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":388,"thread_start_us":213,"threads_started":1}
I20260809 12:21:15.548949 12895 raft_consensus.cc:482] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:21:15.549144 12895 raft_consensus.cc:504] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } } peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } }
I20260809 12:21:15.549746 12895 leader_election.cc:288] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45407bd46e4e4e4c94de4e814f066ede (127.19.183.194:40951), 2006b09388984ccbadee161a62f02309 (127.19.183.195:41153)
W20260809 12:21:15.550882 12304 leader_election.cc:341] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2006b09388984ccbadee161a62f02309 (127.19.183.195:41153): Not found: Tablet not found: eaec7fd228ef4a73941993234c3ee740
W20260809 12:21:15.551095 12305 leader_election.cc:341] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 45407bd46e4e4e4c94de4e814f066ede (127.19.183.194:40951): Not found: Tablet not found: eaec7fd228ef4a73941993234c3ee740
I20260809 12:21:15.551293 12305 leader_election.cc:302] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1469d62f6084bec9ac860cc7d88741a; no voters: 2006b09388984ccbadee161a62f02309, 45407bd46e4e4e4c94de4e814f066ede
I20260809 12:21:15.551636 12895 raft_consensus.cc:2726] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 12:21:15.991956 12896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:15.991281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.194:52993)
0809 12:21:15.991729 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:15.991806 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":241,"threads_started":1}
I20260809 12:21:16.462632 12899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:16.461814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.195:43949)
0809 12:21:16.462367 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:16.462453 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":441,"thread_start_us":337,"threads_started":1}
I20260809 12:21:16.512395 12900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:16.511744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.193:51707)
0809 12:21:16.512161 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:16.512237 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":229,"threads_started":1}
I20260809 12:21:16.993497 12901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:16.992802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.194:38629)
0809 12:21:16.993246 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:16.993322 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":224,"threads_started":1}
I20260809 12:21:17.464299 12902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:17.463534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.195:35177)
0809 12:21:17.463981 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:17.464087 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":209,"threads_started":1}
I20260809 12:21:17.514096 12903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:17.513357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.193:48323)
0809 12:21:17.513833 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:17.513912 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":378,"thread_start_us":219,"threads_started":1}
I20260809 12:21:17.995194 12904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:17.994416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.194:41851)
0809 12:21:17.994929 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:17.995009 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":417,"thread_start_us":238,"threads_started":1}
I20260809 12:21:18.465931 12907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:18.465226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.195:60643)
0809 12:21:18.465675 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:18.465777 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":260,"threads_started":1}
I20260809 12:21:18.515682 12908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:18.514979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.193:34113)
0809 12:21:18.515425 (+ 446us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:18.515511 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":232,"threads_started":1}
I20260809 12:21:18.649641 12912 raft_consensus.cc:482] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:21:18.649899 12912 raft_consensus.cc:504] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } } peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } }
I20260809 12:21:18.650576 12912 leader_election.cc:288] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45407bd46e4e4e4c94de4e814f066ede (127.19.183.194:40951), 2006b09388984ccbadee161a62f02309 (127.19.183.195:41153)
W20260809 12:21:18.651696 12304 leader_election.cc:341] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2006b09388984ccbadee161a62f02309 (127.19.183.195:41153): Not found: Tablet not found: eaec7fd228ef4a73941993234c3ee740
W20260809 12:21:18.651890 12305 leader_election.cc:341] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 45407bd46e4e4e4c94de4e814f066ede (127.19.183.194:40951): Not found: Tablet not found: eaec7fd228ef4a73941993234c3ee740
I20260809 12:21:18.652071 12305 leader_election.cc:302] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1469d62f6084bec9ac860cc7d88741a; no voters: 2006b09388984ccbadee161a62f02309, 45407bd46e4e4e4c94de4e814f066ede
I20260809 12:21:18.652346 12912 raft_consensus.cc:2726] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 12:21:18.996764 12913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:18.996048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.194:44341)
0809 12:21:18.996517 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:18.996598 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":260,"threads_started":1}
I20260809 12:21:19.467597 12915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:19.466885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.195:46375)
0809 12:21:19.467345 (+ 460us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:19.467424 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":248,"threads_started":1}
I20260809 12:21:19.517143 12916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:19.516446 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.193:58495)
0809 12:21:19.516844 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:19.516932 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":208,"threads_started":1}
I20260809 12:21:19.998344 12919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:19.997584 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.194:58911)
0809 12:21:19.998081 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:19.998170 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":409,"thread_start_us":223,"threads_started":1}
I20260809 12:21:20.469205 12920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:20.468435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.195:34073)
0809 12:21:20.468912 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:20.468989 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":227,"threads_started":1}
I20260809 12:21:20.518702 12921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:20.517994 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.193:55165)
0809 12:21:20.518479 (+ 485us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:20.518556 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":395,"thread_start_us":218,"threads_started":1}
I20260809 12:21:20.999940 12922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:20.999311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.194:39455)
0809 12:21:20.999690 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:20.999766 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":196,"threads_started":1}
I20260809 12:21:21.470912 12923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:21.470122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.195:50189)
0809 12:21:21.470561 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:21.470672 (+ 111us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":232,"threads_started":1}
I20260809 12:21:21.520282 12924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:21.519551 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.193:60641)
0809 12:21:21.519991 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:21.520090 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":243,"threads_started":1}
I20260809 12:21:21.738922 12925 raft_consensus.cc:482] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:21:21.739147 12925 raft_consensus.cc:504] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45407bd46e4e4e4c94de4e814f066ede" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 40951 } } peers { permanent_uuid: "2006b09388984ccbadee161a62f02309" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 41153 } } peers { permanent_uuid: "a1469d62f6084bec9ac860cc7d88741a" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 43617 } }
I20260809 12:21:21.739738 12925 leader_election.cc:288] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 45407bd46e4e4e4c94de4e814f066ede (127.19.183.194:40951), 2006b09388984ccbadee161a62f02309 (127.19.183.195:41153)
W20260809 12:21:21.740875 12305 leader_election.cc:341] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 45407bd46e4e4e4c94de4e814f066ede (127.19.183.194:40951): Not found: Tablet not found: eaec7fd228ef4a73941993234c3ee740
W20260809 12:21:21.741087 12304 leader_election.cc:341] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2006b09388984ccbadee161a62f02309 (127.19.183.195:41153): Not found: Tablet not found: eaec7fd228ef4a73941993234c3ee740
I20260809 12:21:21.741236 12304 leader_election.cc:302] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: a1469d62f6084bec9ac860cc7d88741a; no voters: 2006b09388984ccbadee161a62f02309, 45407bd46e4e4e4c94de4e814f066ede
I20260809 12:21:21.741510 12925 raft_consensus.cc:2726] T eaec7fd228ef4a73941993234c3ee740 P a1469d62f6084bec9ac860cc7d88741a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 12:21:22.001452 12926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:22.000787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.194:51369)
0809 12:21:22.001231 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:22.001309 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":227,"threads_started":1}
I20260809 12:21:22.472656 12927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:22.471837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.195:36615)
0809 12:21:22.472398 (+ 561us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:22.472491 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":460,"thread_start_us":318,"threads_started":1}
I20260809 12:21:22.521912 12928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:22.521165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.193:42507)
0809 12:21:22.521613 (+ 448us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:22.521691 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":240,"threads_started":1}
I20260809 12:21:23.003154 12929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:23.002358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.194:58693)
0809 12:21:23.002876 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:23.002959 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":257,"threads_started":1}
I20260809 12:21:23.474310 12930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:23.473519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.195:36965)
0809 12:21:23.474043 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:23.474138 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":430,"thread_start_us":320,"threads_started":1}
I20260809 12:21:23.523471 12931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:23.522806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.193:51549)
0809 12:21:23.523208 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:23.523304 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":210,"threads_started":1}
I20260809 12:21:24.004817 12933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:24.004023 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.194:38063)
0809 12:21:24.004567 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:24.004656 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":440,"thread_start_us":305,"threads_started":1}
I20260809 12:21:24.476097 12934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:24.475376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.195:48127)
0809 12:21:24.475864 (+ 488us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:24.475940 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":233,"threads_started":1}
I20260809 12:21:24.524958 12935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:24.524303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.193:46697)
0809 12:21:24.524718 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:24.524803 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":218,"threads_started":1}
I20260809 12:21:25.006570 12936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:25.005840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.194:55765)
0809 12:21:25.006320 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:25.006401 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":383,"thread_start_us":254,"threads_started":1}
I20260809 12:21:25.477870 12938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:25.477199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.195:48131)
0809 12:21:25.477629 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:25.477706 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":228,"threads_started":1}
I20260809 12:21:25.526515 12941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:25.525795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.193:44807)
0809 12:21:25.526248 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:25.526345 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":238,"threads_started":1}
I20260809 12:21:26.008347 12942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:26.007554 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.194:56663)
0809 12:21:26.008051 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:26.008161 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":403,"thread_start_us":219,"threads_started":1}
I20260809 12:21:26.479544 12943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:26.478825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.195:41167)
0809 12:21:26.479305 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:26.479389 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":241,"threads_started":1}
I20260809 12:21:26.527973 12944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:26.527327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.193:41169)
0809 12:21:26.527735 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:26.527808 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":214,"threads_started":1}
I20260809 12:21:27.010016 12945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:27.009265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.194:54979)
0809 12:21:27.009696 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:27.009787 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":229,"threads_started":1}
I20260809 12:21:27.481132 12950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:27.480411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.195:40699)
0809 12:21:27.480851 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:27.480928 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":230,"threads_started":1}
I20260809 12:21:27.529892 12952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:27.529200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.193:49097)
0809 12:21:27.529632 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:27.529723 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":337,"thread_start_us":226,"threads_started":1}
I20260809 12:21:28.012115 12953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:28.011193 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.194:39363)
0809 12:21:28.011680 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:28.011857 (+ 177us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":379,"thread_start_us":236,"threads_started":1}
I20260809 12:21:28.461184 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 12215
I20260809 12:21:28.476289 12956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:28.462269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38460 (local address 127.19.183.193:43617)
0809 12:21:28.462956 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:28.462967 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:28.463000 (+ 33us) server_negotiation.cc:407] Connection header received
0809 12:21:28.463086 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:28.463105 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:28.463427 (+ 322us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:28.463599 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:28.465058 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:28.465991 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:28.467231 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:28.467802 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:28.467941 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:28.470572 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:28.470592 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:28.470607 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:28.470639 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:28.473367 (+ 2728us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:28.474066 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:28.474075 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:28.474079 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:28.474229 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:28.474603 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:28.474610 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:28.474614 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:28.475017 (+ 403us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:28.475359 (+ 342us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:28.475659 (+ 300us) server_negotiation.cc:299] Negotiation successful
0809 12:21:28.475926 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":290,"threads_started":1}
I20260809 12:21:28.477514 12466 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:28.483042 12960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:28.482239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:39553 (local address 127.19.183.195:51953)
0809 12:21:28.482789 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:28.482896 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.254:39553: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":274,"threads_started":1}
I20260809 12:21:28.564834 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 12215
I20260809 12:21:28.592811 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 12482
I20260809 12:21:28.610615 12963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:28.594331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38732 (local address 127.19.183.194:40951)
0809 12:21:28.594865 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:28.594870 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:28.594887 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:21:28.594942 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:28.594946 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:28.595022 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:28.595124 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:28.597343 (+ 2219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:28.598792 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:28.602199 (+ 3407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:28.602782 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:28.602909 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:28.605541 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:28.605546 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:28.605549 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:28.605581 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:28.607979 (+ 2398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:28.608645 (+ 666us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:28.608651 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:28.608655 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:28.608821 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:28.609200 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:28.609205 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:28.609209 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:28.609469 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:28.609733 (+ 264us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:28.610055 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 12:21:28.610261 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":435,"thread_start_us":306,"threads_started":1}
I20260809 12:21:28.611387 12667 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:28.694036 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 12482
I20260809 12:21:28.719834 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 12679
I20260809 12:21:28.734062 12965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:28.720764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34246 (local address 127.19.183.195:41153)
0809 12:21:28.721338 (+ 574us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:28.721342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:28.721358 (+ 16us) server_negotiation.cc:407] Connection header received
0809 12:21:28.721413 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:28.721417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:28.721482 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:28.721606 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:28.723254 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:28.724066 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:28.725372 (+ 1306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:28.725804 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:28.725941 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:28.728841 (+ 2900us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:28.728845 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:28.728849 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:28.728880 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:28.731389 (+ 2509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:28.732078 (+ 689us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:28.732088 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:28.732090 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:28.732228 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:28.732618 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:28.732625 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:28.732629 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:28.732872 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:28.733179 (+ 307us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:28.733465 (+ 286us) server_negotiation.cc:299] Negotiation successful
0809 12:21:28.733694 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":455,"thread_start_us":286,"threads_started":1}
I20260809 12:21:28.734987 12864 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:28.819346 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 12679
2026-08-09T12:21:28Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (65349 ms)
[----------] 30 tests from SecurityITest (313958 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:21:28 dist-test-slave-39l8 krb5kdc[12973](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:21:28 dist-test-slave-39l8 krb5kdc[12973](info): set up 2 sockets
Aug 09 12:21:28 dist-test-slave-39l8 krb5kdc[12973](info): commencing operation
krb5kdc: starting...
W20260809 12:21:30.137154 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.001s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:21:30 dist-test-slave-39l8 krb5kdc[12973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:21:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:21:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:21:30.452567 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:37509
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:45431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:37509
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:21:30.794584 12994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:30.794937 12994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:30.795114 12994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:30.802043 12994 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:21:30.802183 12994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:30.802291 12994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:30.802392 12994 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:21:30.802479 12994 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:21:30.811937 12994 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:45431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:37509
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:37509
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:30.813308 12994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:30.814631 12994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:30.826834 13001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:30.826869 13004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:30.826956 13000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:30.827560 12994 server_base.cc:1034] running on GCE node
I20260809 12:21:30.828325 12994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:30.829571 12994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:30.830821 12994 hybrid_clock.cc:648] HybridClock initialized: now 1786278090830758 us; error 56 us; skew 500 ppm
I20260809 12:21:30.831157 12994 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:21:30 dist-test-slave-39l8 krb5kdc[12973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278090, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:21:30.839841 12994 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:21:30.841349 12994 webserver.cc:460] Webserver started at http://127.19.183.254:45727/ using document root <none> and password file <none>
I20260809 12:21:30.842056 12994 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:30.842211 12994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:30.842528 12994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:30.845532 12994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "8f578066c4cb4bcaaa064e5704536cb1"
format_stamp: "Formatted at 2026-08-09 12:21:30 on dist-test-slave-39l8"
server_key: "5911e23d17b4f0f55db0d82a54a478ac"
server_key_iv: "fc4538fe933287dc62eb8dc92f893be2"
server_key_version: "encryptionkey@0"
I20260809 12:21:30.846290 12994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "8f578066c4cb4bcaaa064e5704536cb1"
format_stamp: "Formatted at 2026-08-09 12:21:30 on dist-test-slave-39l8"
server_key: "5911e23d17b4f0f55db0d82a54a478ac"
server_key_iv: "fc4538fe933287dc62eb8dc92f893be2"
server_key_version: "encryptionkey@0"
I20260809 12:21:30.851292 12994 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260809 12:21:30.854671 13017 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:30.855967 12994 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:21:30.856184 12994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "8f578066c4cb4bcaaa064e5704536cb1"
format_stamp: "Formatted at 2026-08-09 12:21:30 on dist-test-slave-39l8"
server_key: "5911e23d17b4f0f55db0d82a54a478ac"
server_key_iv: "fc4538fe933287dc62eb8dc92f893be2"
server_key_version: "encryptionkey@0"
I20260809 12:21:30.856401 12994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:30.879968 12994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:30.884055 12994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:30.884366 12994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:30.903422 12994 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:37509
I20260809 12:21:30.903446 13091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:37509 every 8 connection(s)
I20260809 12:21:30.904922 12994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:21:30.908931 13092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:30.915108 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 12994
I20260809 12:21:30.915318 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:21:30.915997 20191 external_mini_cluster.cc:1428] Setting key 733bc8173d9edadf779af2007e8e5286
I20260809 12:21:30.920293 13092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1: Bootstrap starting.
I20260809 12:21:30.923553 13092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:30.924717 13092 log.cc:824] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:30.927848 13092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1: No bootstrap required, opened a new log
Aug 09 12:21:30 dist-test-slave-39l8 krb5kdc[12973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:21:30.936612 13092 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f578066c4cb4bcaaa064e5704536cb1" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 37509 } }
I20260809 12:21:30.937024 13092 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:30.937148 13092 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f578066c4cb4bcaaa064e5704536cb1, State: Initialized, Role: FOLLOWER
I20260809 12:21:30.937784 13092 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f578066c4cb4bcaaa064e5704536cb1" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 37509 } }
I20260809 12:21:30.938050 13092 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:21:30.938238 13092 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:21:30.938424 13092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:30.940323 13097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:30.918202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49480 (local address 127.19.183.254:37509)
0809 12:21:30.919140 (+ 938us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:30.919153 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:30.919195 (+ 42us) server_negotiation.cc:407] Connection header received
0809 12:21:30.921016 (+ 1821us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:30.921044 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:30.921430 (+ 386us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:30.921951 (+ 521us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:30.923678 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:30.925151 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:30.926961 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:30.927654 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:30.927783 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:30.932615 (+ 4832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:30.932626 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:30.932642 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:30.932683 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:30.935982 (+ 3299us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:30.936748 (+ 766us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:30.936755 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:30.936761 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:30.936922 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:30.937447 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:30.937451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:30.937453 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:30.937973 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:30.938312 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:30.938826 (+ 514us) server_negotiation.cc:299] Negotiation successful
0809 12:21:30.939239 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":692,"thread_start_us":453,"threads_started":1}
I20260809 12:21:30.941095 13092 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f578066c4cb4bcaaa064e5704536cb1" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 37509 } }
I20260809 12:21:30.941583 13092 leader_election.cc:302] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f578066c4cb4bcaaa064e5704536cb1; no voters:
I20260809 12:21:30.942189 13092 leader_election.cc:288] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:21:30.942345 13099 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:21:30.943759 13099 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1 [term 1 LEADER]: Becoming Leader. State: Replica: 8f578066c4cb4bcaaa064e5704536cb1, State: Running, Role: LEADER
I20260809 12:21:30.944331 13099 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f578066c4cb4bcaaa064e5704536cb1" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 37509 } }
I20260809 12:21:30.944456 13092 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:21:30.949462 13101 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f578066c4cb4bcaaa064e5704536cb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f578066c4cb4bcaaa064e5704536cb1" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 37509 } } }
I20260809 12:21:30.949452 13102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f578066c4cb4bcaaa064e5704536cb1. Latest consensus state: current_term: 1 leader_uuid: "8f578066c4cb4bcaaa064e5704536cb1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f578066c4cb4bcaaa064e5704536cb1" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 37509 } } }
I20260809 12:21:30.949910 13101 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1 [sys.catalog]: This master's current role is: LEADER
I20260809 12:21:30.949982 13102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1 [sys.catalog]: This master's current role is: LEADER
I20260809 12:21:30.952523 13118 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:21:30.957152 13118 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:21:30.966899 13118 catalog_manager.cc:1347] Generated new cluster ID: ef1d8e4aaa754d5c8563acc51a44a7ca
I20260809 12:21:30.967023 13118 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:21:30.992676 13118 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:21:30.994374 13118 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:21:31.008268 13118 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1: Generated new TSK 0
I20260809 12:21:31.009141 13118 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
I20260809 12:21:31.112931 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:37509
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:45431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:21:31.483626 13133 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:31.483884 13133 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:31.484074 13133 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:31.491613 13133 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:31.491743 13133 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:31.491916 13133 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:21:31.501461 13133 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:45431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:37509
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:31.502681 13133 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:31.504166 13133 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:31.517248 13141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:31.517311 13146 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:31.517336 13140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:31.518087 13133 server_base.cc:1034] running on GCE node
I20260809 12:21:31.518774 13133 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:31.519727 13133 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:31.520929 13133 hybrid_clock.cc:648] HybridClock initialized: now 1786278091520879 us; error 39 us; skew 500 ppm
I20260809 12:21:31.521256 13133 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:21:31 dist-test-slave-39l8 krb5kdc[12973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278091, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:21:31.524546 13133 init.cc:377] Logged in from keytab as kudu/127.19.183.193@KRBTEST.COM (short username kudu)
I20260809 12:21:31.526011 13133 webserver.cc:460] Webserver started at http://127.19.183.193:37931/ using document root <none> and password file <none>
I20260809 12:21:31.526655 13133 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:31.526826 13133 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:31.527120 13133 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:31.529891 13133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "9c379e8943494fdc8651f2d9478320bb"
format_stamp: "Formatted at 2026-08-09 12:21:31 on dist-test-slave-39l8"
server_key: "a3f1e6fc9860a16ea540daa4f35e2582"
server_key_iv: "61b66fbd43cd729d6ffd90ca7e469861"
server_key_version: "encryptionkey@0"
I20260809 12:21:31.530593 13133 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "9c379e8943494fdc8651f2d9478320bb"
format_stamp: "Formatted at 2026-08-09 12:21:31 on dist-test-slave-39l8"
server_key: "a3f1e6fc9860a16ea540daa4f35e2582"
server_key_iv: "61b66fbd43cd729d6ffd90ca7e469861"
server_key_version: "encryptionkey@0"
I20260809 12:21:31.535691 13133 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 12:21:31.539013 13159 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:31.540297 13133 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:21:31.540472 13133 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "9c379e8943494fdc8651f2d9478320bb"
format_stamp: "Formatted at 2026-08-09 12:21:31 on dist-test-slave-39l8"
server_key: "a3f1e6fc9860a16ea540daa4f35e2582"
server_key_iv: "61b66fbd43cd729d6ffd90ca7e469861"
server_key_version: "encryptionkey@0"
I20260809 12:21:31.540668 13133 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:31.560707 13133 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:31.564937 13133 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:31.565246 13133 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:31.566226 13133 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:21:31.567636 13133 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:21:31.567737 13133 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:31.567888 13133 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:21:31.567960 13133 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:31.602669 13133 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:37417
I20260809 12:21:31.602783 13344 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:37417 every 8 connection(s)
I20260809 12:21:31.604101 13133 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:21:31.612815 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 13133
I20260809 12:21:31.613039 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:21:31.613771 20191 external_mini_cluster.cc:1428] Setting key 89dbccd6b24a8b448f6af08ed9740fa8
Aug 09 12:21:31 dist-test-slave-39l8 krb5kdc[12973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278091, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:21:31.625900 13348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:31.606734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:48423 (local address 127.19.183.254:37509)
0809 12:21:31.607175 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:31.607180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:31.608284 (+ 1104us) server_negotiation.cc:407] Connection header received
0809 12:21:31.610162 (+ 1878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:31.610168 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:31.610262 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:31.610385 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:31.612004 (+ 1619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:31.613113 (+ 1109us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:31.614749 (+ 1636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:31.615207 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:31.615320 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:31.618823 (+ 3503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:31.618832 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:31.618839 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:31.618886 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:31.622460 (+ 3574us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:31.623257 (+ 797us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:31.623265 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:31.623271 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:31.623432 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:31.623859 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:31.623866 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:31.623872 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:31.624273 (+ 401us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:31.624433 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:31.625105 (+ 672us) server_negotiation.cc:299] Negotiation successful
0809 12:21:31.625389 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":221,"threads_started":1}
I20260809 12:21:31.626538 13349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:31.607281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:37509 (local address 127.19.183.193:48423)
0809 12:21:31.608085 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:31.608135 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:31.609882 (+ 1747us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:31.610636 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:31.610644 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:31.611249 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:21:31.611812 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:31.611830 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:31.613273 (+ 1443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:31.613277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:31.614591 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:31.614605 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:31.615462 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:31.615470 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:31.615682 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:31.616374 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:21:31.616402 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:21:31.618553 (+ 2151us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:21:31.622618 (+ 4065us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:21:31.622628 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:21:31.622641 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:21:31.623139 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:21:31.623551 (+ 412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:21:31.623554 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:21:31.623556 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:21:31.623740 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:21:31.624395 (+ 655us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:21:31.624401 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:21:31.624718 (+ 317us) client_negotiation.cc:769] Sending connection context
0809 12:21:31.624940 (+ 222us) client_negotiation.cc:240] Negotiation successful
0809 12:21:31.625376 (+ 436us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":359,"threads_started":1}
I20260809 12:21:31.628180 13345 heartbeater.cc:347] Connected to a master server at 127.19.183.254:37509
I20260809 12:21:31.628532 13345 heartbeater.cc:464] Registering TS with master...
I20260809 12:21:31.629427 13345 heartbeater.cc:511] Master 127.19.183.254:37509 requested a full tablet report, sending...
I20260809 12:21:31.632138 13036 ts_manager.cc:194] Registered new tserver with Master: 9c379e8943494fdc8651f2d9478320bb (127.19.183.193:37417)
I20260809 12:21:31.634321 13036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:48423
WARNING: no policy specified for kudu/127.19.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
I20260809 12:21:31.688269 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:37509
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:45431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:21:32.042303 13353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:32.042599 13353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:32.042754 13353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:32.050482 13353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:32.050601 13353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:32.050786 13353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:21:32.060932 13353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:45431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:37509
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:32.062283 13353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:32.063684 13353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:32.077080 13361 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:32.077103 13367 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:32.077049 13365 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:32.077839 13353 server_base.cc:1034] running on GCE node
I20260809 12:21:32.078473 13353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:32.079540 13353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:32.080773 13353 hybrid_clock.cc:648] HybridClock initialized: now 1786278092080729 us; error 56 us; skew 500 ppm
I20260809 12:21:32.081115 13353 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:21:32 dist-test-slave-39l8 krb5kdc[12973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278092, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:21:32.084527 13353 init.cc:377] Logged in from keytab as kudu/127.19.183.194@KRBTEST.COM (short username kudu)
I20260809 12:21:32.086015 13353 webserver.cc:460] Webserver started at http://127.19.183.194:34441/ using document root <none> and password file <none>
I20260809 12:21:32.086848 13353 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:32.086961 13353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:32.087252 13353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:32.090210 13353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "ce87194e75f743b3ae17837bc5fe850d"
format_stamp: "Formatted at 2026-08-09 12:21:32 on dist-test-slave-39l8"
server_key: "317cb0d3f53059331647bf922cfa0489"
server_key_iv: "4b6ec2382720d1b9457a798cc64cedcf"
server_key_version: "encryptionkey@0"
I20260809 12:21:32.091085 13353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "ce87194e75f743b3ae17837bc5fe850d"
format_stamp: "Formatted at 2026-08-09 12:21:32 on dist-test-slave-39l8"
server_key: "317cb0d3f53059331647bf922cfa0489"
server_key_iv: "4b6ec2382720d1b9457a798cc64cedcf"
server_key_version: "encryptionkey@0"
I20260809 12:21:32.096452 13353 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 12:21:32.100006 13377 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:32.101300 13353 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:21:32.101478 13353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "ce87194e75f743b3ae17837bc5fe850d"
format_stamp: "Formatted at 2026-08-09 12:21:32 on dist-test-slave-39l8"
server_key: "317cb0d3f53059331647bf922cfa0489"
server_key_iv: "4b6ec2382720d1b9457a798cc64cedcf"
server_key_version: "encryptionkey@0"
I20260809 12:21:32.101711 13353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:32.125259 13353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:32.129869 13353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:32.130201 13353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:32.131294 13353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:21:32.132695 13353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:21:32.132809 13353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:32.132972 13353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:21:32.133049 13353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:32.168125 13353 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:37301
I20260809 12:21:32.168184 13550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:37301 every 8 connection(s)
I20260809 12:21:32.169484 13353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:21:32.172326 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 13353
I20260809 12:21:32.172533 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:21:32.173009 20191 external_mini_cluster.cc:1428] Setting key 1b569af9df1a73193c6d95b806d02ea3
Aug 09 12:21:32 dist-test-slave-39l8 krb5kdc[12973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278092, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:21:32.190377 13554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:32.172278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:52009 (local address 127.19.183.254:37509)
0809 12:21:32.175129 (+ 2851us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:32.175135 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:32.175158 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:21:32.176145 (+ 987us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:32.176150 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:32.176236 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:32.176379 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:32.177881 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:32.178577 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:32.180197 (+ 1620us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:32.180671 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:32.180811 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:32.184366 (+ 3555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:32.184371 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:32.184374 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:32.184406 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:32.187135 (+ 2729us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:32.187954 (+ 819us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:32.187970 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:32.187972 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:32.188108 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:32.188593 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:32.188598 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:32.188600 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:32.188879 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:32.189043 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:32.189759 (+ 716us) server_negotiation.cc:299] Negotiation successful
0809 12:21:32.189985 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2702,"thread_start_us":2585,"threads_started":1}
I20260809 12:21:32.191453 13557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:32.172947 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:37509 (local address 127.19.183.194:52009)
0809 12:21:32.173903 (+ 956us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:32.173944 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:32.175872 (+ 1928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:32.176620 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:32.176630 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:32.177146 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:21:32.177707 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:32.177724 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:32.178788 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:32.178793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:32.180005 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:32.180019 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:32.180906 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:32.180914 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:32.181158 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:32.181890 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:21:32.181912 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:21:32.184141 (+ 2229us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:21:32.187282 (+ 3141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:21:32.187291 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:21:32.187308 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:21:32.187799 (+ 491us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:21:32.188247 (+ 448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:21:32.188253 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:21:32.188256 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:21:32.188460 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:21:32.189025 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:21:32.189033 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:21:32.189385 (+ 352us) client_negotiation.cc:769] Sending connection context
0809 12:21:32.189664 (+ 279us) client_negotiation.cc:240] Negotiation successful
0809 12:21:32.190221 (+ 557us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":681,"thread_start_us":390,"threads_started":1}
I20260809 12:21:32.193014 13551 heartbeater.cc:347] Connected to a master server at 127.19.183.254:37509
I20260809 12:21:32.193364 13551 heartbeater.cc:464] Registering TS with master...
I20260809 12:21:32.194319 13551 heartbeater.cc:511] Master 127.19.183.254:37509 requested a full tablet report, sending...
I20260809 12:21:32.196092 13036 ts_manager.cc:194] Registered new tserver with Master: ce87194e75f743b3ae17837bc5fe850d (127.19.183.194:37301)
I20260809 12:21:32.197381 13036 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:52009
WARNING: no policy specified for kudu/127.19.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
I20260809 12:21:32.248454 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:37509
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:45431
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:21:32.596762 13560 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:32.597028 13560 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:32.597131 13560 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:32.604411 13560 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:32.604514 13560 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:32.604717 13560 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:21:32.614289 13560 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:45431
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:37509
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:32.615535 13560 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:32.616860 13560 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:32.630163 13567 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:32.630110 13568 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:32.630317 13571 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:32.631276 13560 server_base.cc:1034] running on GCE node
I20260809 12:21:32.631888 13560 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:32.632859 13560 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:32.634083 13560 hybrid_clock.cc:648] HybridClock initialized: now 1786278092634024 us; error 50 us; skew 500 ppm
I20260809 12:21:32.634395 13560 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:21:32 dist-test-slave-39l8 krb5kdc[12973](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278092, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:21:32.637290 13345 heartbeater.cc:503] Master 127.19.183.254:37509 was elected leader, sending a full tablet report...
I20260809 12:21:32.637567 13560 init.cc:377] Logged in from keytab as kudu/127.19.183.195@KRBTEST.COM (short username kudu)
I20260809 12:21:32.639272 13560 webserver.cc:460] Webserver started at http://127.19.183.195:36917/ using document root <none> and password file <none>
I20260809 12:21:32.639956 13560 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:32.640060 13560 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:32.640328 13560 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:32.643234 13560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "8063ce2f2a874cdeb5bd8f9f3ce6d357"
format_stamp: "Formatted at 2026-08-09 12:21:32 on dist-test-slave-39l8"
server_key: "b48a0b3450efee2a18f070730dd8b3bd"
server_key_iv: "d34ac877faed476458468c5be1664281"
server_key_version: "encryptionkey@0"
I20260809 12:21:32.643976 13560 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "8063ce2f2a874cdeb5bd8f9f3ce6d357"
format_stamp: "Formatted at 2026-08-09 12:21:32 on dist-test-slave-39l8"
server_key: "b48a0b3450efee2a18f070730dd8b3bd"
server_key_iv: "d34ac877faed476458468c5be1664281"
server_key_version: "encryptionkey@0"
I20260809 12:21:32.649292 13560 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.005s
I20260809 12:21:32.652726 13582 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:32.654012 13560 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 12:21:32.654152 13560 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "8063ce2f2a874cdeb5bd8f9f3ce6d357"
format_stamp: "Formatted at 2026-08-09 12:21:32 on dist-test-slave-39l8"
server_key: "b48a0b3450efee2a18f070730dd8b3bd"
server_key_iv: "d34ac877faed476458468c5be1664281"
server_key_version: "encryptionkey@0"
I20260809 12:21:32.654316 13560 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:32.689446 13560 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:32.693758 13560 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:32.694005 13560 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:32.694981 13560 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:21:32.696305 13560 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:21:32.696376 13560 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:32.696475 13560 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:21:32.696517 13560 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:32.730618 13560 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:43993
I20260809 12:21:32.730670 13761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:43993 every 8 connection(s)
I20260809 12:21:32.731945 13560 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:21:32.742314 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 13560
I20260809 12:21:32.742507 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:21:32.743083 20191 external_mini_cluster.cc:1428] Setting key 9ea0211e7ac5c40032da5a5927f29997
Aug 09 12:21:32 dist-test-slave-39l8 krb5kdc[12973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278092, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:21:32.752455 13764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:32.734548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:42743 (local address 127.19.183.254:37509)
0809 12:21:32.735036 (+ 488us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:32.735041 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:32.735971 (+ 930us) server_negotiation.cc:407] Connection header received
0809 12:21:32.737758 (+ 1787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:32.737781 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:32.737873 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:32.737993 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:32.739550 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:32.740312 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:32.741863 (+ 1551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:32.742525 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:32.742679 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:32.746858 (+ 4179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:32.746863 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:32.746866 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:32.746898 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:32.749459 (+ 2561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:32.750164 (+ 705us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:32.750167 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:32.750170 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:32.750307 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:32.750687 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:32.750691 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:32.750693 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:32.751003 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:32.751124 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:32.751766 (+ 642us) server_negotiation.cc:299] Negotiation successful
0809 12:21:32.752012 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":265,"threads_started":1}
I20260809 12:21:32.753712 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:32.735104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:37509 (local address 127.19.183.195:42743)
0809 12:21:32.735798 (+ 694us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:32.735848 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:32.737486 (+ 1638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:32.738242 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:32.738249 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:32.738831 (+ 582us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:21:32.739381 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:32.739397 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:32.740484 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:32.740488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:32.741697 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:32.741709 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:32.742812 (+ 1103us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:32.742822 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:32.743043 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:32.743798 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:21:32.743825 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:21:32.746496 (+ 2671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:21:32.749619 (+ 3123us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:21:32.749626 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:21:32.749638 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:21:32.750062 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:21:32.750402 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:21:32.750406 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:21:32.750408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:21:32.750578 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:21:32.751115 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:21:32.751121 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:21:32.751416 (+ 295us) client_negotiation.cc:769] Sending connection context
0809 12:21:32.751647 (+ 231us) client_negotiation.cc:240] Negotiation successful
0809 12:21:32.752192 (+ 545us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":483,"thread_start_us":270,"threads_started":1}
I20260809 12:21:32.755378 13762 heartbeater.cc:347] Connected to a master server at 127.19.183.254:37509
I20260809 12:21:32.755704 13762 heartbeater.cc:464] Registering TS with master...
I20260809 12:21:32.756508 13762 heartbeater.cc:511] Master 127.19.183.254:37509 requested a full tablet report, sending...
I20260809 12:21:32.758369 13035 ts_manager.cc:194] Registered new tserver with Master: 8063ce2f2a874cdeb5bd8f9f3ce6d357 (127.19.183.195:43993)
I20260809 12:21:32.759426 13035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:42743
I20260809 12:21:32.770771 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:21:32.788213 13764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:32.775395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40616 (local address 127.19.183.254:37509)
0809 12:21:32.775599 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:32.775607 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:32.775995 (+ 388us) server_negotiation.cc:407] Connection header received
0809 12:21:32.776208 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:32.776215 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:32.776285 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:32.776398 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:32.777840 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:32.778597 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:32.779942 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:32.780335 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:32.780490 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:32.783060 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:32.783065 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:32.783068 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:32.783106 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:32.785515 (+ 2409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:32.786215 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:32.786220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:32.786224 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:32.786352 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:32.786750 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:32.786756 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:32.786762 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:32.787075 (+ 313us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:32.787357 (+ 282us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:32.787668 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 12:21:32.787888 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 12:21:32.792034 13035 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40616:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:21:32.798838 13035 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:21:32.839835 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:32.811058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59830 (local address 127.19.183.194:37301)
0809 12:21:32.829330 (+ 18272us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:32.829341 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:32.829377 (+ 36us) server_negotiation.cc:407] Connection header received
0809 12:21:32.829491 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:32.829501 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:32.829774 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:32.832299 (+ 2525us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:32.832572 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:32.833717 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:32.836981 (+ 3264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:32.838465 (+ 1484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:32.838688 (+ 223us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:32.838886 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:32.839284 (+ 398us) server_negotiation.cc:299] Negotiation successful
0809 12:21:32.839464 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18091,"thread_start_us":17967,"threads_started":1}
I20260809 12:21:32.840164 13783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:32.811551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34736 (local address 127.19.183.193:37417)
0809 12:21:32.831274 (+ 19723us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:32.831281 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:32.831304 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:21:32.831383 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:32.831388 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:32.831628 (+ 240us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:32.831829 (+ 201us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:32.832958 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:32.834516 (+ 1558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:32.837295 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:32.838753 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:32.838940 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:32.839133 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:32.839664 (+ 531us) server_negotiation.cc:299] Negotiation successful
0809 12:21:32.839836 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19567,"thread_start_us":19382,"threads_started":1}
I20260809 12:21:32.839900 13780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:32.810813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:37301 (local address 127.0.0.1:59830)
0809 12:21:32.823477 (+ 12664us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:32.823518 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:32.826058 (+ 2540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:32.831618 (+ 5560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:32.831626 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:32.831668 (+ 42us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:32.832171 (+ 503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:32.832189 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:32.833925 (+ 1736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:32.833930 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:32.836791 (+ 2861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:32.836807 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:32.838745 (+ 1938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:32.838751 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:32.838967 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:32.839075 (+ 108us) client_negotiation.cc:769] Sending connection context
0809 12:21:32.839220 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 12:21:32.839432 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":12368,"mutex_wait_us":156,"thread_start_us":12194,"threads_started":1}
I20260809 12:21:32.840250 13786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:32.812289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54982 (local address 127.19.183.195:43993)
0809 12:21:32.829733 (+ 17444us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:32.829742 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:32.831191 (+ 1449us) server_negotiation.cc:407] Connection header received
0809 12:21:32.831300 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:32.831306 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:32.831508 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:32.831702 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:32.832783 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:32.834859 (+ 2076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:32.836925 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:32.838145 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:32.838329 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:32.838520 (+ 191us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:32.839583 (+ 1063us) server_negotiation.cc:299] Negotiation successful
0809 12:21:32.839772 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17270,"thread_start_us":17128,"threads_started":1}
I20260809 12:21:32.840791 13784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:32.811188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:37417 (local address 127.0.0.1:34736)
0809 12:21:32.830228 (+ 19040us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:32.830252 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:32.830384 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:32.832087 (+ 1703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:32.832101 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:32.832124 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:32.832763 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:32.832778 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:32.834678 (+ 1900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:32.834683 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:32.837109 (+ 2426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:32.837123 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:32.838931 (+ 1808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:32.838937 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:32.839220 (+ 283us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:32.839252 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 12:21:32.839342 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 12:21:32.839488 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18880,"mutex_wait_us":133,"thread_start_us":19467,"threads_started":1}
I20260809 12:21:32.841562 13782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:32.811582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:43993 (local address 127.0.0.1:54982)
0809 12:21:32.830937 (+ 19355us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:32.830952 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:32.831064 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:32.832016 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:32.832022 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:32.832074 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:32.832575 (+ 501us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:32.832585 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:32.835029 (+ 2444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:32.835034 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:32.836776 (+ 1742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:32.836789 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:32.838671 (+ 1882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:32.838677 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:32.839067 (+ 390us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:32.839096 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 12:21:32.839230 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 12:21:32.839432 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19233,"mutex_wait_us":259,"thread_start_us":18324,"threads_started":1}
I20260809 12:21:32.844550 13245 tablet_service.cc:1467] Processing CreateTablet for tablet f663da13e04e4edb8e2e547650123004 (DEFAULT_TABLE table=test-table [id=7712812a1b254106a68951d59ee191fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:21:32.844552 13444 tablet_service.cc:1467] Processing CreateTablet for tablet f663da13e04e4edb8e2e547650123004 (DEFAULT_TABLE table=test-table [id=7712812a1b254106a68951d59ee191fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:21:32.844962 13664 tablet_service.cc:1467] Processing CreateTablet for tablet f663da13e04e4edb8e2e547650123004 (DEFAULT_TABLE table=test-table [id=7712812a1b254106a68951d59ee191fe]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:21:32.846122 13245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f663da13e04e4edb8e2e547650123004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:32.846141 13444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f663da13e04e4edb8e2e547650123004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:32.846307 13664 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f663da13e04e4edb8e2e547650123004. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:32.857856 13787 tablet_bootstrap.cc:492] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357: Bootstrap starting.
I20260809 12:21:32.861645 13787 tablet_bootstrap.cc:654] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:32.862463 13791 tablet_bootstrap.cc:492] T f663da13e04e4edb8e2e547650123004 P 9c379e8943494fdc8651f2d9478320bb: Bootstrap starting.
I20260809 12:21:32.863054 13787 log.cc:824] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:32.863404 13788 tablet_bootstrap.cc:492] T f663da13e04e4edb8e2e547650123004 P ce87194e75f743b3ae17837bc5fe850d: Bootstrap starting.
I20260809 12:21:32.866257 13787 tablet_bootstrap.cc:492] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357: No bootstrap required, opened a new log
I20260809 12:21:32.866545 13787 ts_tablet_manager.cc:1397] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.003s
I20260809 12:21:32.866770 13791 tablet_bootstrap.cc:654] T f663da13e04e4edb8e2e547650123004 P 9c379e8943494fdc8651f2d9478320bb: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:32.868073 13791 log.cc:824] T f663da13e04e4edb8e2e547650123004 P 9c379e8943494fdc8651f2d9478320bb: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:32.869699 13788 tablet_bootstrap.cc:654] T f663da13e04e4edb8e2e547650123004 P ce87194e75f743b3ae17837bc5fe850d: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:32.870633 13788 log.cc:824] T f663da13e04e4edb8e2e547650123004 P ce87194e75f743b3ae17837bc5fe850d: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:32.873638 13788 tablet_bootstrap.cc:492] T f663da13e04e4edb8e2e547650123004 P ce87194e75f743b3ae17837bc5fe850d: No bootstrap required, opened a new log
I20260809 12:21:32.873914 13788 ts_tablet_manager.cc:1397] T f663da13e04e4edb8e2e547650123004 P ce87194e75f743b3ae17837bc5fe850d: Time spent bootstrapping tablet: real 0.011s user 0.000s sys 0.005s
I20260809 12:21:32.880656 13787 raft_consensus.cc:348] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce87194e75f743b3ae17837bc5fe850d" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 37301 } } peers { permanent_uuid: "9c379e8943494fdc8651f2d9478320bb" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37417 } } peers { permanent_uuid: "8063ce2f2a874cdeb5bd8f9f3ce6d357" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 43993 } }
I20260809 12:21:32.881067 13787 raft_consensus.cc:374] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:32.881186 13787 raft_consensus.cc:729] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8063ce2f2a874cdeb5bd8f9f3ce6d357, State: Initialized, Role: FOLLOWER
I20260809 12:21:32.881759 13788 raft_consensus.cc:348] T f663da13e04e4edb8e2e547650123004 P ce87194e75f743b3ae17837bc5fe850d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce87194e75f743b3ae17837bc5fe850d" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 37301 } } peers { permanent_uuid: "9c379e8943494fdc8651f2d9478320bb" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37417 } } peers { permanent_uuid: "8063ce2f2a874cdeb5bd8f9f3ce6d357" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 43993 } }
I20260809 12:21:32.881829 13787 consensus_queue.cc:260] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce87194e75f743b3ae17837bc5fe850d" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 37301 } } peers { permanent_uuid: "9c379e8943494fdc8651f2d9478320bb" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37417 } } peers { permanent_uuid: "8063ce2f2a874cdeb5bd8f9f3ce6d357" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 43993 } }
I20260809 12:21:32.882107 13788 raft_consensus.cc:374] T f663da13e04e4edb8e2e547650123004 P ce87194e75f743b3ae17837bc5fe850d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:32.882203 13788 raft_consensus.cc:729] T f663da13e04e4edb8e2e547650123004 P ce87194e75f743b3ae17837bc5fe850d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ce87194e75f743b3ae17837bc5fe850d, State: Initialized, Role: FOLLOWER
I20260809 12:21:32.883005 13791 tablet_bootstrap.cc:492] T f663da13e04e4edb8e2e547650123004 P 9c379e8943494fdc8651f2d9478320bb: No bootstrap required, opened a new log
I20260809 12:21:32.882876 13788 consensus_queue.cc:260] T f663da13e04e4edb8e2e547650123004 P ce87194e75f743b3ae17837bc5fe850d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce87194e75f743b3ae17837bc5fe850d" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 37301 } } peers { permanent_uuid: "9c379e8943494fdc8651f2d9478320bb" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37417 } } peers { permanent_uuid: "8063ce2f2a874cdeb5bd8f9f3ce6d357" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 43993 } }
I20260809 12:21:32.883365 13791 ts_tablet_manager.cc:1397] T f663da13e04e4edb8e2e547650123004 P 9c379e8943494fdc8651f2d9478320bb: Time spent bootstrapping tablet: real 0.021s user 0.006s sys 0.013s
I20260809 12:21:32.884121 13551 heartbeater.cc:503] Master 127.19.183.254:37509 was elected leader, sending a full tablet report...
I20260809 12:21:32.885973 13788 ts_tablet_manager.cc:1428] T f663da13e04e4edb8e2e547650123004 P ce87194e75f743b3ae17837bc5fe850d: Time spent starting tablet: real 0.012s user 0.011s sys 0.002s
I20260809 12:21:32.888948 13762 heartbeater.cc:503] Master 127.19.183.254:37509 was elected leader, sending a full tablet report...
I20260809 12:21:32.890228 13787 ts_tablet_manager.cc:1428] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357: Time spent starting tablet: real 0.024s user 0.016s sys 0.008s
I20260809 12:21:32.894764 13791 raft_consensus.cc:348] T f663da13e04e4edb8e2e547650123004 P 9c379e8943494fdc8651f2d9478320bb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce87194e75f743b3ae17837bc5fe850d" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 37301 } } peers { permanent_uuid: "9c379e8943494fdc8651f2d9478320bb" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37417 } } peers { permanent_uuid: "8063ce2f2a874cdeb5bd8f9f3ce6d357" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 43993 } }
I20260809 12:21:32.895095 13791 raft_consensus.cc:374] T f663da13e04e4edb8e2e547650123004 P 9c379e8943494fdc8651f2d9478320bb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:32.895169 13791 raft_consensus.cc:729] T f663da13e04e4edb8e2e547650123004 P 9c379e8943494fdc8651f2d9478320bb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9c379e8943494fdc8651f2d9478320bb, State: Initialized, Role: FOLLOWER
I20260809 12:21:32.895685 13791 consensus_queue.cc:260] T f663da13e04e4edb8e2e547650123004 P 9c379e8943494fdc8651f2d9478320bb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce87194e75f743b3ae17837bc5fe850d" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 37301 } } peers { permanent_uuid: "9c379e8943494fdc8651f2d9478320bb" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37417 } } peers { permanent_uuid: "8063ce2f2a874cdeb5bd8f9f3ce6d357" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 43993 } }
I20260809 12:21:32.896922 13791 ts_tablet_manager.cc:1428] T f663da13e04e4edb8e2e547650123004 P 9c379e8943494fdc8651f2d9478320bb: Time spent starting tablet: real 0.013s user 0.013s sys 0.000s
W20260809 12:21:32.921700 13552 tablet.cc:2369] T f663da13e04e4edb8e2e547650123004 P ce87194e75f743b3ae17837bc5fe850d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:21:32.983671 13763 tablet.cc:2369] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:21:33.106667 13347 tablet.cc:2369] T f663da13e04e4edb8e2e547650123004 P 9c379e8943494fdc8651f2d9478320bb: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:21:33.248159 13797 raft_consensus.cc:482] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:21:33.248561 13797 raft_consensus.cc:504] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce87194e75f743b3ae17837bc5fe850d" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 37301 } } peers { permanent_uuid: "9c379e8943494fdc8651f2d9478320bb" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37417 } } peers { permanent_uuid: "8063ce2f2a874cdeb5bd8f9f3ce6d357" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 43993 } }
I20260809 12:21:33.250377 13797 leader_election.cc:288] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ce87194e75f743b3ae17837bc5fe850d (127.19.183.194:37301), 9c379e8943494fdc8651f2d9478320bb (127.19.183.193:37417)
I20260809 12:21:33.256778 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:33.250919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:50403 (local address 127.19.183.194:37301)
0809 12:21:33.251132 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:33.251138 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:33.251161 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:21:33.251259 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:33.251271 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:33.251331 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:33.251435 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:33.251906 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:33.252941 (+ 1035us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:33.254801 (+ 1860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:33.255950 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:33.256064 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:33.256117 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:33.256480 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 12:21:33.256570 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260809 12:21:33.256867 13766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:33.250616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:37301 (local address 127.19.183.195:50403)
0809 12:21:33.250935 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:33.250956 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:33.251119 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:33.251463 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:33.251468 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:33.251501 (+ 33us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:33.251707 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:33.251718 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:33.253070 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:33.253076 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:33.254617 (+ 1541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:33.254631 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:33.256110 (+ 1479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:33.256116 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:33.256307 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:33.256343 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 12:21:33.256449 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 12:21:33.256585 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190}
I20260809 12:21:33.257746 13783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:33.250821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:48693 (local address 127.19.183.193:37417)
0809 12:21:33.251083 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:33.251093 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:33.251775 (+ 682us) server_negotiation.cc:407] Connection header received
0809 12:21:33.251945 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:33.251948 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:33.252020 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:33.252088 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:33.252688 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:33.253630 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:33.255665 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:33.256763 (+ 1098us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:33.256929 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:33.257007 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:33.257440 (+ 433us) server_negotiation.cc:299] Negotiation successful
0809 12:21:33.257544 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260809 12:21:33.257833 13802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:33.250815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:37417 (local address 127.19.183.195:48693)
0809 12:21:33.251663 (+ 848us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:33.251677 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:33.251785 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:33.252144 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:33.252148 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:33.252176 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:33.252539 (+ 363us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:33.252550 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:33.253781 (+ 1231us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:33.253784 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:33.255496 (+ 1712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:33.255512 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:33.257143 (+ 1631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:33.257150 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:33.257280 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:33.257301 (+ 21us) client_negotiation.cc:769] Sending connection context
0809 12:21:33.257378 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 12:21:33.257502 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":736,"thread_start_us":539,"threads_started":1}
I20260809 12:21:33.258049 13478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f663da13e04e4edb8e2e547650123004" candidate_uuid: "8063ce2f2a874cdeb5bd8f9f3ce6d357" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce87194e75f743b3ae17837bc5fe850d" is_pre_election: true
I20260809 12:21:33.258643 13478 raft_consensus.cc:2454] T f663da13e04e4edb8e2e547650123004 P ce87194e75f743b3ae17837bc5fe850d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8063ce2f2a874cdeb5bd8f9f3ce6d357 in term 0.
I20260809 12:21:33.259034 13277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f663da13e04e4edb8e2e547650123004" candidate_uuid: "8063ce2f2a874cdeb5bd8f9f3ce6d357" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c379e8943494fdc8651f2d9478320bb" is_pre_election: true
I20260809 12:21:33.259362 13586 leader_election.cc:302] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8063ce2f2a874cdeb5bd8f9f3ce6d357, ce87194e75f743b3ae17837bc5fe850d; no voters:
I20260809 12:21:33.259662 13277 raft_consensus.cc:2454] T f663da13e04e4edb8e2e547650123004 P 9c379e8943494fdc8651f2d9478320bb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8063ce2f2a874cdeb5bd8f9f3ce6d357 in term 0.
I20260809 12:21:33.259817 13797 raft_consensus.cc:2781] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:21:33.259966 13797 raft_consensus.cc:482] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:21:33.260083 13797 raft_consensus.cc:3037] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:33.262400 13797 raft_consensus.cc:504] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce87194e75f743b3ae17837bc5fe850d" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 37301 } } peers { permanent_uuid: "9c379e8943494fdc8651f2d9478320bb" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37417 } } peers { permanent_uuid: "8063ce2f2a874cdeb5bd8f9f3ce6d357" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 43993 } }
I20260809 12:21:33.263098 13797 leader_election.cc:288] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357 [CANDIDATE]: Term 1 election: Requested vote from peers ce87194e75f743b3ae17837bc5fe850d (127.19.183.194:37301), 9c379e8943494fdc8651f2d9478320bb (127.19.183.193:37417)
I20260809 12:21:33.263518 13478 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f663da13e04e4edb8e2e547650123004" candidate_uuid: "8063ce2f2a874cdeb5bd8f9f3ce6d357" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ce87194e75f743b3ae17837bc5fe850d"
I20260809 12:21:33.263595 13277 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f663da13e04e4edb8e2e547650123004" candidate_uuid: "8063ce2f2a874cdeb5bd8f9f3ce6d357" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9c379e8943494fdc8651f2d9478320bb"
I20260809 12:21:33.263726 13478 raft_consensus.cc:3037] T f663da13e04e4edb8e2e547650123004 P ce87194e75f743b3ae17837bc5fe850d [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:33.263839 13277 raft_consensus.cc:3037] T f663da13e04e4edb8e2e547650123004 P 9c379e8943494fdc8651f2d9478320bb [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:33.265990 13478 raft_consensus.cc:2454] T f663da13e04e4edb8e2e547650123004 P ce87194e75f743b3ae17837bc5fe850d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8063ce2f2a874cdeb5bd8f9f3ce6d357 in term 1.
I20260809 12:21:33.266098 13277 raft_consensus.cc:2454] T f663da13e04e4edb8e2e547650123004 P 9c379e8943494fdc8651f2d9478320bb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8063ce2f2a874cdeb5bd8f9f3ce6d357 in term 1.
I20260809 12:21:33.266462 13586 leader_election.cc:302] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8063ce2f2a874cdeb5bd8f9f3ce6d357, ce87194e75f743b3ae17837bc5fe850d; no voters:
I20260809 12:21:33.266803 13797 raft_consensus.cc:2781] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:21:33.267227 13797 raft_consensus.cc:686] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357 [term 1 LEADER]: Becoming Leader. State: Replica: 8063ce2f2a874cdeb5bd8f9f3ce6d357, State: Running, Role: LEADER
I20260809 12:21:33.267684 13797 consensus_queue.cc:237] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce87194e75f743b3ae17837bc5fe850d" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 37301 } } peers { permanent_uuid: "9c379e8943494fdc8651f2d9478320bb" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37417 } } peers { permanent_uuid: "8063ce2f2a874cdeb5bd8f9f3ce6d357" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 43993 } }
I20260809 12:21:33.272475 13035 catalog_manager.cc:5495] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8063ce2f2a874cdeb5bd8f9f3ce6d357 (127.19.183.195). New cstate: current_term: 1 leader_uuid: "8063ce2f2a874cdeb5bd8f9f3ce6d357" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ce87194e75f743b3ae17837bc5fe850d" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 37301 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9c379e8943494fdc8651f2d9478320bb" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37417 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8063ce2f2a874cdeb5bd8f9f3ce6d357" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 43993 } health_report { overall_health: HEALTHY } } }
I20260809 12:21:33.294181 13786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:33.288618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54984 (local address 127.19.183.195:43993)
0809 12:21:33.288842 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:33.288849 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:33.289233 (+ 384us) server_negotiation.cc:407] Connection header received
0809 12:21:33.289384 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:33.289387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:33.289479 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:33.289597 (+ 118us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:21:33.289978 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:33.290670 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:33.292214 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:33.292598 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:33.292690 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:33.293159 (+ 469us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:21:33.293622 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:21:33.293737 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:33.293816 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 12:21:33.293879 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 12:21:33.302106 13478 raft_consensus.cc:1261] T f663da13e04e4edb8e2e547650123004 P ce87194e75f743b3ae17837bc5fe850d [term 1 FOLLOWER]: Refusing update from remote peer 8063ce2f2a874cdeb5bd8f9f3ce6d357: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:21:33.302142 13277 raft_consensus.cc:1261] T f663da13e04e4edb8e2e547650123004 P 9c379e8943494fdc8651f2d9478320bb [term 1 FOLLOWER]: Refusing update from remote peer 8063ce2f2a874cdeb5bd8f9f3ce6d357: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:21:33.303117 13803 consensus_queue.cc:1035] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9c379e8943494fdc8651f2d9478320bb" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37417 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:21:33.303488 13797 consensus_queue.cc:1035] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ce87194e75f743b3ae17837bc5fe850d" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 37301 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:21:33.314381 13808 mvcc.cc:205] Tried to move back new op lower bound from 7316595070155296768 to 7316595070028230656. Current Snapshot: MvccSnapshot[applied={T|T < 7316595070155296768}]
I20260809 12:21:33.329550 13035 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:21:33.329797 13035 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:21:33.333207 13035 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8f578066c4cb4bcaaa064e5704536cb1: Sending DeleteTablet for 3 replicas of tablet f663da13e04e4edb8e2e547650123004
I20260809 12:21:33.334463 13444 tablet_service.cc:1514] Processing DeleteTablet for tablet f663da13e04e4edb8e2e547650123004 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:21:33 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:59830
I20260809 12:21:33.334488 13245 tablet_service.cc:1514] Processing DeleteTablet for tablet f663da13e04e4edb8e2e547650123004 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:21:33 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:34736
I20260809 12:21:33.334488 13664 tablet_service.cc:1514] Processing DeleteTablet for tablet f663da13e04e4edb8e2e547650123004 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:21:33 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:54982
I20260809 12:21:33.336252 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 13133
I20260809 12:21:33.350311 13825 tablet_replica.cc:330] T f663da13e04e4edb8e2e547650123004 P 9c379e8943494fdc8651f2d9478320bb: stopping tablet replica
I20260809 12:21:33.350761 13825 raft_consensus.cc:2229] T f663da13e04e4edb8e2e547650123004 P 9c379e8943494fdc8651f2d9478320bb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:21:33.351583 13825 raft_consensus.cc:2258] T f663da13e04e4edb8e2e547650123004 P 9c379e8943494fdc8651f2d9478320bb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:21:33.354074 13825 ts_tablet_manager.cc:1905] T f663da13e04e4edb8e2e547650123004 P 9c379e8943494fdc8651f2d9478320bb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:21:33.354766 13826 tablet_replica.cc:330] T f663da13e04e4edb8e2e547650123004 P ce87194e75f743b3ae17837bc5fe850d: stopping tablet replica
I20260809 12:21:33.354140 13824 tablet_replica.cc:330] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357: stopping tablet replica
I20260809 12:21:33.355314 13824 raft_consensus.cc:2229] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:21:33.355834 13826 raft_consensus.cc:2229] T f663da13e04e4edb8e2e547650123004 P ce87194e75f743b3ae17837bc5fe850d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:21:33.357033 13824 raft_consensus.cc:2258] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:21:33.357087 13826 raft_consensus.cc:2258] T f663da13e04e4edb8e2e547650123004 P ce87194e75f743b3ae17837bc5fe850d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:21:33.359414 13824 ts_tablet_manager.cc:1905] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:21:33.360595 13826 ts_tablet_manager.cc:1905] T f663da13e04e4edb8e2e547650123004 P ce87194e75f743b3ae17837bc5fe850d: Deleting tablet data with delete state TABLET_DATA_DELETED
Aug 09 12:21:33 dist-test-slave-39l8 krb5kdc[12973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.193@KRBTEST.COM
I20260809 12:21:33.367554 13826 ts_tablet_manager.cc:1918] T f663da13e04e4edb8e2e547650123004 P ce87194e75f743b3ae17837bc5fe850d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:21:33.367735 13826 log.cc:1196] T f663da13e04e4edb8e2e547650123004 P ce87194e75f743b3ae17837bc5fe850d: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/f663da13e04e4edb8e2e547650123004
I20260809 12:21:33.368163 13824 ts_tablet_manager.cc:1918] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:21:33.368350 13826 ts_tablet_manager.cc:1939] T f663da13e04e4edb8e2e547650123004 P ce87194e75f743b3ae17837bc5fe850d: Deleting consensus metadata
I20260809 12:21:33.368481 13824 log.cc:1196] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/f663da13e04e4edb8e2e547650123004
I20260809 12:21:33.369141 13824 ts_tablet_manager.cc:1939] T f663da13e04e4edb8e2e547650123004 P 8063ce2f2a874cdeb5bd8f9f3ce6d357: Deleting consensus metadata
I20260809 12:21:33.370404 13020 catalog_manager.cc:4841] TS ce87194e75f743b3ae17837bc5fe850d (127.19.183.194:37301): tablet f663da13e04e4edb8e2e547650123004 (table test-table [id=7712812a1b254106a68951d59ee191fe]) successfully deleted
I20260809 12:21:33.370826 13783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:33.337892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34738 (local address 127.19.183.193:37417)
0809 12:21:33.338125 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:33.338133 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:33.343221 (+ 5088us) server_negotiation.cc:407] Connection header received
0809 12:21:33.344306 (+ 1085us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:33.344315 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:33.344410 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:33.344512 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:33.349321 (+ 4809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:33.350672 (+ 1351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:33.354686 (+ 4014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:33.355414 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:33.355537 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:33.362847 (+ 7310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:33.362862 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:33.362880 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:33.362932 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:33.366901 (+ 3969us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:33.367875 (+ 974us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:33.367884 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:33.367888 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:33.368066 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:33.368616 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:33.368622 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:33.368627 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:33.369147 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:33.369450 (+ 303us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:33.370148 (+ 698us) server_negotiation.cc:299] Negotiation successful
0809 12:21:33.370388 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 12:21:33.370848 13021 catalog_manager.cc:4841] TS 8063ce2f2a874cdeb5bd8f9f3ce6d357 (127.19.183.195:43993): tablet f663da13e04e4edb8e2e547650123004 (table test-table [id=7712812a1b254106a68951d59ee191fe]) successfully deleted
I20260809 12:21:33.372462 13337 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:33.462972 13825 ts_tablet_manager.cc:1918] T f663da13e04e4edb8e2e547650123004 P 9c379e8943494fdc8651f2d9478320bb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:21:33.463511 13825 log.cc:1196] T f663da13e04e4edb8e2e547650123004 P 9c379e8943494fdc8651f2d9478320bb: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/f663da13e04e4edb8e2e547650123004
I20260809 12:21:33.463646 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 13133
I20260809 12:21:33.491321 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 13353
W20260809 12:21:33.491317 13018 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.19.183.193:37417 (error 108)
I20260809 12:21:33.493566 13831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:33.492686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:37417 (local address 127.0.0.1:34740)
0809 12:21:33.493193 (+ 507us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:33.493323 (+ 130us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:37417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":376,"thread_start_us":270,"threads_started":1}
W20260809 12:21:33.493848 13018 catalog_manager.cc:4584] TS 9c379e8943494fdc8651f2d9478320bb (127.19.183.193:37417): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f663da13e04e4edb8e2e547650123004: Network error: Client connection negotiation failed: client connection to 127.19.183.193:37417: connect: Connection refused (error 111)
Aug 09 12:21:33 dist-test-slave-39l8 krb5kdc[12973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.194@KRBTEST.COM
I20260809 12:21:33.509317 13781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:33.492669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59842 (local address 127.19.183.194:37301)
0809 12:21:33.492893 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:33.492900 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:33.492921 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:21:33.492992 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:33.492997 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:33.493077 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:33.493184 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:33.494828 (+ 1644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:33.495925 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:33.497120 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:33.497747 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:33.497889 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:33.502404 (+ 4515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:33.502425 (+ 21us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:33.502446 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:33.502501 (+ 55us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:33.506243 (+ 3742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:33.507081 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:33.507090 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:33.507094 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:33.507267 (+ 173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:33.507632 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:33.507637 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:33.507640 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:33.508146 (+ 506us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:33.508454 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:33.508702 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 12:21:33.508933 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 12:21:33.509999 13545 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:33.547657 13831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:33.547084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:37417 (local address 127.0.0.1:34746)
0809 12:21:33.547327 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:33.547460 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:37417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":86}
I20260809 12:21:33.548940 13831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:33.548507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:37417 (local address 127.0.0.1:34756)
0809 12:21:33.548674 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:33.548760 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:37417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 12:21:33.602054 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 13353
I20260809 12:21:33.613950 13831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:33.612946 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:37417 (local address 127.0.0.1:34758)
0809 12:21:33.613354 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:33.613589 (+ 235us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:37417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115}
I20260809 12:21:33.615692 13831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:33.615148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:37417 (local address 127.0.0.1:34770)
0809 12:21:33.615394 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:33.615492 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:37417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128}
I20260809 12:21:33.642405 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 13560
Aug 09 12:21:33 dist-test-slave-39l8 krb5kdc[12973](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278090, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.195@KRBTEST.COM
I20260809 12:21:33.663856 13786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:33.643319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54994 (local address 127.19.183.195:43993)
0809 12:21:33.643574 (+ 255us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:33.643580 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:33.643601 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:21:33.643741 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:33.643746 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:33.643838 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:33.643952 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:33.645526 (+ 1574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:33.647003 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:33.648455 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:33.649297 (+ 842us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:33.649451 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:33.654365 (+ 4914us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:33.654383 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:33.654405 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:33.654465 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:33.659300 (+ 4835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:33.660226 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:33.660237 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:33.660242 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:33.660490 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:33.661757 (+ 1267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:33.661763 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:33.661767 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:33.662391 (+ 624us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:33.662793 (+ 402us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:33.663158 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 12:21:33.663414 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101}
I20260809 12:21:33.664780 13753 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:33.696884 13831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:33.696370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:37417 (local address 127.0.0.1:34782)
0809 12:21:33.696613 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:33.696728 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:37417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260809 12:21:33.698035 13831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:33.697639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:37417 (local address 127.0.0.1:34794)
0809 12:21:33.697834 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:33.697905 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.193:37417: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":110}
I20260809 12:21:33.790001 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 13560
I20260809 12:21:33.816308 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 12994
I20260809 12:21:33.817817 13084 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:33.930866 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 12994
2026-08-09T12:21:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (5112 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T12:21:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:21:34Z Disabled control of system clock
I20260809 12:21:34.034366 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:35113
--webserver_interface=127.19.183.254
--webserver_port=0
--builtin_ntp_servers=127.19.183.212:37757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:35113
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 12:21:34.385789 13839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:34.386107 13839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:34.386251 13839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:34.393638 13839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:21:34.393744 13839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:34.393836 13839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:34.393944 13839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:21:34.394024 13839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:21:34.403185 13839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:37757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:35113
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:35113
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:34.404508 13839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:34.406323 13839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:34.418346 13846 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:34.418401 13851 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:34.418319 13847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:34.419158 13839 server_base.cc:1034] running on GCE node
I20260809 12:21:34.419967 13839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:34.421326 13839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:34.422564 13839 hybrid_clock.cc:648] HybridClock initialized: now 1786278094422523 us; error 55 us; skew 500 ppm
I20260809 12:21:34.422964 13839 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:21:34.425240 13839 webserver.cc:460] Webserver started at http://127.19.183.254:34617/ using document root <none> and password file <none>
I20260809 12:21:34.426002 13839 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:34.426147 13839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:34.426476 13839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:34.429622 13839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "5234cf93dc664bda819e10c4d261cc00"
format_stamp: "Formatted at 2026-08-09 12:21:34 on dist-test-slave-39l8"
server_key: "cf8ff1dc5d6ef221492d781867ea4878"
server_key_iv: "e0c85bc9e77974428e155b8255d2901a"
server_key_version: "encryptionkey@0"
I20260809 12:21:34.430514 13839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "5234cf93dc664bda819e10c4d261cc00"
format_stamp: "Formatted at 2026-08-09 12:21:34 on dist-test-slave-39l8"
server_key: "cf8ff1dc5d6ef221492d781867ea4878"
server_key_iv: "e0c85bc9e77974428e155b8255d2901a"
server_key_version: "encryptionkey@0"
I20260809 12:21:34.435892 13839 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 12:21:34.439395 13861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:34.440706 13839 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:21:34.440902 13839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "5234cf93dc664bda819e10c4d261cc00"
format_stamp: "Formatted at 2026-08-09 12:21:34 on dist-test-slave-39l8"
server_key: "cf8ff1dc5d6ef221492d781867ea4878"
server_key_iv: "e0c85bc9e77974428e155b8255d2901a"
server_key_version: "encryptionkey@0"
I20260809 12:21:34.441138 13839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:34.463168 13839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:34.464022 13839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:34.464334 13839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:34.482867 13839 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:35113
I20260809 12:21:34.482842 13934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:35113 every 8 connection(s)
I20260809 12:21:34.484177 13839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:21:34.484659 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 13839
I20260809 12:21:34.484846 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:21:34.485553 20191 external_mini_cluster.cc:1428] Setting key e5a5dbf67744d80b630752324dc06252
I20260809 12:21:34.488574 13936 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:34.499298 13941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:34.487538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50680 (local address 127.19.183.254:35113)
0809 12:21:34.488538 (+ 1000us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:34.488548 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:34.488592 (+ 44us) server_negotiation.cc:407] Connection header received
0809 12:21:34.490204 (+ 1612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:34.490225 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:34.490639 (+ 414us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:34.491004 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:34.491642 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:34.492942 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:34.494976 (+ 2034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:34.495627 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:34.495754 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:34.497020 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:34.497028 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:34.497041 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:34.497594 (+ 553us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:34.497672 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:34.497692 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:34.497774 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:34.498018 (+ 244us) server_negotiation.cc:299] Negotiation successful
0809 12:21:34.498356 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":379,"threads_started":1}
I20260809 12:21:34.500757 13936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00: Bootstrap starting.
I20260809 12:21:34.504038 13936 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:34.504944 13936 log.cc:824] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:34.507507 13936 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00: No bootstrap required, opened a new log
I20260809 12:21:34.515375 13936 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5234cf93dc664bda819e10c4d261cc00" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35113 } }
I20260809 12:21:34.515699 13936 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:34.515801 13936 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5234cf93dc664bda819e10c4d261cc00, State: Initialized, Role: FOLLOWER
I20260809 12:21:34.516294 13936 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5234cf93dc664bda819e10c4d261cc00" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35113 } }
I20260809 12:21:34.516491 13936 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:21:34.516585 13936 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:21:34.516721 13936 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:34.519269 13936 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5234cf93dc664bda819e10c4d261cc00" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35113 } }
I20260809 12:21:34.519732 13936 leader_election.cc:302] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5234cf93dc664bda819e10c4d261cc00; no voters:
I20260809 12:21:34.520303 13936 leader_election.cc:288] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:21:34.520505 13945 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:21:34.521345 13945 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00 [term 1 LEADER]: Becoming Leader. State: Replica: 5234cf93dc664bda819e10c4d261cc00, State: Running, Role: LEADER
I20260809 12:21:34.521819 13945 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5234cf93dc664bda819e10c4d261cc00" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35113 } }
I20260809 12:21:34.522483 13936 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:21:34.524991 13946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5234cf93dc664bda819e10c4d261cc00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5234cf93dc664bda819e10c4d261cc00" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35113 } } }
I20260809 12:21:34.525156 13947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5234cf93dc664bda819e10c4d261cc00. Latest consensus state: current_term: 1 leader_uuid: "5234cf93dc664bda819e10c4d261cc00" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5234cf93dc664bda819e10c4d261cc00" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 35113 } } }
I20260809 12:21:34.525468 13946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00 [sys.catalog]: This master's current role is: LEADER
I20260809 12:21:34.525525 13947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00 [sys.catalog]: This master's current role is: LEADER
I20260809 12:21:34.527158 13957 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:21:34.531682 13957 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:21:34.540283 13957 catalog_manager.cc:1347] Generated new cluster ID: 1f50695018cd4007b1e70d9777e35750
I20260809 12:21:34.540390 13957 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:21:34.570941 13957 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:21:34.572871 13957 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:21:34.590284 13957 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00: Generated new TSK 0
I20260809 12:21:34.591296 13957 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:21:34.600256 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35113
--builtin_ntp_servers=127.19.183.212:37757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 12:21:34.951921 13977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:34.952245 13977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:34.952334 13977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:34.959672 13977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:34.959776 13977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:34.959882 13977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:21:34.968930 13977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:37757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:34.970206 13977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:34.972136 13977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:34.985447 13986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:34.985396 13987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:34.985492 13989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:34.986327 13977 server_base.cc:1034] running on GCE node
I20260809 12:21:34.987015 13977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:34.987931 13977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:34.989146 13977 hybrid_clock.cc:648] HybridClock initialized: now 1786278094989103 us; error 29 us; skew 500 ppm
I20260809 12:21:34.989461 13977 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:21:34.991976 13977 webserver.cc:460] Webserver started at http://127.19.183.193:41167/ using document root <none> and password file <none>
I20260809 12:21:34.992710 13977 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:34.992828 13977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:34.993206 13977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:34.996145 13977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "2b7cd5a49046400ca9400368c9617f7e"
format_stamp: "Formatted at 2026-08-09 12:21:34 on dist-test-slave-39l8"
server_key: "18def4f3e4855fef94020590997b0841"
server_key_iv: "0906eb96b4da04e347fc81e316850f88"
server_key_version: "encryptionkey@0"
I20260809 12:21:34.996937 13977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "2b7cd5a49046400ca9400368c9617f7e"
format_stamp: "Formatted at 2026-08-09 12:21:34 on dist-test-slave-39l8"
server_key: "18def4f3e4855fef94020590997b0841"
server_key_iv: "0906eb96b4da04e347fc81e316850f88"
server_key_version: "encryptionkey@0"
I20260809 12:21:35.002003 13977 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260809 12:21:35.005515 13999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:35.006955 13977 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:21:35.007133 13977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "2b7cd5a49046400ca9400368c9617f7e"
format_stamp: "Formatted at 2026-08-09 12:21:34 on dist-test-slave-39l8"
server_key: "18def4f3e4855fef94020590997b0841"
server_key_iv: "0906eb96b4da04e347fc81e316850f88"
server_key_version: "encryptionkey@0"
I20260809 12:21:35.007346 13977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:35.033972 13977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:35.034891 13977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:35.035223 13977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:35.036218 13977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:21:35.037703 13977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:21:35.037820 13977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:35.037943 13977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:21:35.038056 13977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:35.073118 13977 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:41787
I20260809 12:21:35.073230 14167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:41787 every 8 connection(s)
I20260809 12:21:35.074512 13977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:21:35.083750 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 13977
I20260809 12:21:35.083956 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:21:35.084507 20191 external_mini_cluster.cc:1428] Setting key 32f4ded9ceaf75c5be282fbab351226b
I20260809 12:21:35.087683 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35113
--builtin_ntp_servers=127.19.183.212:37757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 12:21:35.088768 14171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:35.077370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:44225 (local address 127.19.183.254:35113)
0809 12:21:35.077896 (+ 526us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:35.077904 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:35.079249 (+ 1345us) server_negotiation.cc:407] Connection header received
0809 12:21:35.081015 (+ 1766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:35.081020 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:35.081089 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:35.081192 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:35.082227 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:35.083125 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:35.085492 (+ 2367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:35.085955 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:35.086058 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:35.087743 (+ 1685us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:35.087750 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:35.087754 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:35.087779 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:35.087831 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:35.087850 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:35.087951 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:35.088205 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 12:21:35.088316 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":272,"threads_started":1}
I20260809 12:21:35.089586 14172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:35.077985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35113 (local address 127.19.183.193:44225)
0809 12:21:35.078968 (+ 983us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:35.079013 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:35.080677 (+ 1664us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:35.081413 (+ 736us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:35.081421 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:35.081508 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:21:35.082067 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:35.082083 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:35.083280 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:35.083284 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:35.085361 (+ 2077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:35.085374 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:35.086208 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:35.086216 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:35.086546 (+ 330us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:35.087395 (+ 849us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:21:35.087398 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:21:35.087438 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:21:35.087521 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:21:35.087987 (+ 466us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:21:35.087997 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:21:35.088003 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 12:21:35.088156 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 12:21:35.088447 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":478,"thread_start_us":275,"threads_started":1}
I20260809 12:21:35.091286 14168 heartbeater.cc:347] Connected to a master server at 127.19.183.254:35113
I20260809 12:21:35.091600 14168 heartbeater.cc:464] Registering TS with master...
I20260809 12:21:35.092360 14168 heartbeater.cc:511] Master 127.19.183.254:35113 requested a full tablet report, sending...
I20260809 12:21:35.094498 13882 ts_manager.cc:194] Registered new tserver with Master: 2b7cd5a49046400ca9400368c9617f7e (127.19.183.193:41787)
I20260809 12:21:35.095973 13882 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.183.193:44225
W20260809 12:21:35.452353 14174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:35.452876 14174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:35.453071 14174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:35.467082 14174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:35.467201 14174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:35.467411 14174 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:21:35.477299 14174 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:37757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:35.478775 14174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:35.480696 14174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:35.495406 14180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:35.495500 14179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:35.496191 14182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:35.496665 14174 server_base.cc:1034] running on GCE node
I20260809 12:21:35.497262 14174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:35.498252 14174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:35.499505 14174 hybrid_clock.cc:648] HybridClock initialized: now 1786278095499457 us; error 38 us; skew 500 ppm
I20260809 12:21:35.499858 14174 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:21:35.502159 14174 webserver.cc:460] Webserver started at http://127.19.183.194:34381/ using document root <none> and password file <none>
I20260809 12:21:35.502978 14174 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:35.503134 14174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:35.503484 14174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:35.506294 14174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "57ae388037bc4e789133ff2e9b1a35d3"
format_stamp: "Formatted at 2026-08-09 12:21:35 on dist-test-slave-39l8"
server_key: "3e7844316d4e66c27f5f51ac43aa3aec"
server_key_iv: "60d784a53610e3f2b2b0e86696ed5f2a"
server_key_version: "encryptionkey@0"
I20260809 12:21:35.507239 14174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "57ae388037bc4e789133ff2e9b1a35d3"
format_stamp: "Formatted at 2026-08-09 12:21:35 on dist-test-slave-39l8"
server_key: "3e7844316d4e66c27f5f51ac43aa3aec"
server_key_iv: "60d784a53610e3f2b2b0e86696ed5f2a"
server_key_version: "encryptionkey@0"
I20260809 12:21:35.512552 14174 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.002s
I20260809 12:21:35.516070 14189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:35.517411 14174 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:21:35.517613 14174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "57ae388037bc4e789133ff2e9b1a35d3"
format_stamp: "Formatted at 2026-08-09 12:21:35 on dist-test-slave-39l8"
server_key: "3e7844316d4e66c27f5f51ac43aa3aec"
server_key_iv: "60d784a53610e3f2b2b0e86696ed5f2a"
server_key_version: "encryptionkey@0"
I20260809 12:21:35.517851 14174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:35.530934 14174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:35.531739 14174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:35.532075 14174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:35.533016 14174 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:21:35.534514 14174 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:21:35.534668 14174 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:35.534889 14174 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:21:35.534998 14174 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:35.578431 14174 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:46125
I20260809 12:21:35.578495 14357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:46125 every 8 connection(s)
I20260809 12:21:35.579792 14174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:21:35.585280 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 14174
I20260809 12:21:35.585472 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:21:35.586018 20191 external_mini_cluster.cc:1428] Setting key 14526e1b47644ce855757b86698010c6
I20260809 12:21:35.589109 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35113
--builtin_ntp_servers=127.19.183.212:37757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 12:21:35.593864 14171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:35.582399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:34371 (local address 127.19.183.254:35113)
0809 12:21:35.582572 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:35.582576 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:35.583943 (+ 1367us) server_negotiation.cc:407] Connection header received
0809 12:21:35.586249 (+ 2306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:35.586254 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:35.586323 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:35.586409 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:35.587670 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:35.588560 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:35.590995 (+ 2435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:35.591361 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:35.591456 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:35.593129 (+ 1673us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:35.593134 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:35.593137 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:35.593156 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:35.593188 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:35.593200 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:35.593266 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:35.593527 (+ 261us) server_negotiation.cc:299] Negotiation successful
0809 12:21:35.593627 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 12:21:35.595309 14363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:35.582843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35113 (local address 127.19.183.194:34371)
0809 12:21:35.583788 (+ 945us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:35.583823 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:35.585939 (+ 2116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:35.586676 (+ 737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:35.586684 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:35.586826 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:21:35.587501 (+ 675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:35.587517 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:35.588735 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:35.588742 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:35.590849 (+ 2107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:35.590868 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:35.591583 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:35.591592 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:35.591908 (+ 316us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:35.592748 (+ 840us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:21:35.592754 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:21:35.592815 (+ 61us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:21:35.592916 (+ 101us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:21:35.593298 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:21:35.593312 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:21:35.593322 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 12:21:35.593504 (+ 182us) client_negotiation.cc:240] Negotiation successful
0809 12:21:35.593909 (+ 405us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":287,"threads_started":1}
I20260809 12:21:35.596779 14359 heartbeater.cc:347] Connected to a master server at 127.19.183.254:35113
I20260809 12:21:35.597103 14359 heartbeater.cc:464] Registering TS with master...
I20260809 12:21:35.598017 14359 heartbeater.cc:511] Master 127.19.183.254:35113 requested a full tablet report, sending...
I20260809 12:21:35.599534 13882 ts_manager.cc:194] Registered new tserver with Master: 57ae388037bc4e789133ff2e9b1a35d3 (127.19.183.194:46125)
I20260809 12:21:35.600299 13882 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.183.194:34371
W20260809 12:21:35.935432 14365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:35.935689 14365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:35.935776 14365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:35.942783 14365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:35.942869 14365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:35.942965 14365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:21:35.951821 14365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:37757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:35113
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:35.953089 14365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:35.954871 14365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:35.967671 14373 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:35.967641 14375 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:35.967708 14372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:35.968823 14365 server_base.cc:1034] running on GCE node
I20260809 12:21:35.969403 14365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:35.970309 14365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:35.971544 14365 hybrid_clock.cc:648] HybridClock initialized: now 1786278095971500 us; error 54 us; skew 500 ppm
I20260809 12:21:35.971846 14365 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:21:35.977452 14365 webserver.cc:460] Webserver started at http://127.19.183.195:38853/ using document root <none> and password file <none>
I20260809 12:21:35.978137 14365 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:35.978272 14365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:35.978590 14365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:35.981479 14365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "d97903fd90534946ae3b1aee8d90525e"
format_stamp: "Formatted at 2026-08-09 12:21:35 on dist-test-slave-39l8"
server_key: "bbe35c5818a9090c5b30756344ca6a19"
server_key_iv: "4a3bf358c3db93f9b1b14e6b37d6ca78"
server_key_version: "encryptionkey@0"
I20260809 12:21:35.982311 14365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "d97903fd90534946ae3b1aee8d90525e"
format_stamp: "Formatted at 2026-08-09 12:21:35 on dist-test-slave-39l8"
server_key: "bbe35c5818a9090c5b30756344ca6a19"
server_key_iv: "4a3bf358c3db93f9b1b14e6b37d6ca78"
server_key_version: "encryptionkey@0"
I20260809 12:21:35.987331 14365 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 12:21:35.990618 14390 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:35.991895 14365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:21:35.992105 14365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "d97903fd90534946ae3b1aee8d90525e"
format_stamp: "Formatted at 2026-08-09 12:21:35 on dist-test-slave-39l8"
server_key: "bbe35c5818a9090c5b30756344ca6a19"
server_key_iv: "4a3bf358c3db93f9b1b14e6b37d6ca78"
server_key_version: "encryptionkey@0"
I20260809 12:21:35.992331 14365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:36.002555 14365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:36.003361 14365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:36.003670 14365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:36.004587 14365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:21:36.005946 14365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:21:36.006068 14365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:36.006210 14365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:21:36.006299 14365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:36.043359 14365 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:35233
I20260809 12:21:36.043411 14560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:35233 every 8 connection(s)
I20260809 12:21:36.044869 14365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:21:36.053961 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 14365
I20260809 12:21:36.054160 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:21:36.054668 20191 external_mini_cluster.cc:1428] Setting key 91c9767232832326711a5f496ee04033
I20260809 12:21:36.058310 14171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:36.047673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:49957 (local address 127.19.183.254:35113)
0809 12:21:36.047870 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:36.047874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:36.049295 (+ 1421us) server_negotiation.cc:407] Connection header received
0809 12:21:36.051161 (+ 1866us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:36.051166 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:36.051261 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:36.051377 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:36.052522 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.053317 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.055064 (+ 1747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.055646 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.055809 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:36.057455 (+ 1646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:36.057489 (+ 34us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:36.057497 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:36.057522 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:36.057566 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:36.057583 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:36.057690 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:36.057965 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 12:21:36.058045 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 12:21:36.059377 14564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:36.048199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:35113 (local address 127.19.183.195:49957)
0809 12:21:36.049109 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:36.049151 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:36.050879 (+ 1728us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:36.051623 (+ 744us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:36.051632 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:36.051722 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:21:36.052344 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:36.052360 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.053490 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.053494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:36.054923 (+ 1429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:36.054935 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.055939 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.055947 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:36.056298 (+ 351us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:36.057135 (+ 837us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:21:36.057138 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:21:36.057178 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:21:36.057259 (+ 81us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:21:36.057727 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:21:36.057740 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:21:36.057748 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 12:21:36.057935 (+ 187us) client_negotiation.cc:240] Negotiation successful
0809 12:21:36.058239 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":485,"thread_start_us":274,"threads_started":1}
I20260809 12:21:36.060683 14561 heartbeater.cc:347] Connected to a master server at 127.19.183.254:35113
I20260809 12:21:36.060992 14561 heartbeater.cc:464] Registering TS with master...
I20260809 12:21:36.061702 14561 heartbeater.cc:511] Master 127.19.183.254:35113 requested a full tablet report, sending...
I20260809 12:21:36.063094 13882 ts_manager.cc:194] Registered new tserver with Master: d97903fd90534946ae3b1aee8d90525e (127.19.183.195:35233)
I20260809 12:21:36.063905 13882 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.183.195:49957
I20260809 12:21:36.069972 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:21:36.080961 14171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:36.074348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50692 (local address 127.19.183.254:35113)
0809 12:21:36.074545 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:36.074552 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:36.075004 (+ 452us) server_negotiation.cc:407] Connection header received
0809 12:21:36.075200 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:36.075205 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:36.075290 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:36.075392 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:36.075893 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.076566 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.078569 (+ 2003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.079177 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.079280 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:36.080274 (+ 994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:36.080279 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:36.080283 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:36.080301 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:36.080333 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:36.080344 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:36.080414 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:36.080536 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 12:21:36.080609 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 12:21:36.084653 13882 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:50692:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:21:36.091492 13882 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:21:36.098891 14168 heartbeater.cc:503] Master 127.19.183.254:35113 was elected leader, sending a full tablet report...
I20260809 12:21:36.131821 14584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:36.104046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:41787 (local address 127.0.0.1:36594)
0809 12:21:36.122749 (+ 18703us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:36.122773 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:36.123419 (+ 646us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:36.124442 (+ 1023us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:36.124447 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:36.124470 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:36.124795 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:36.124809 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.126513 (+ 1704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.126518 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:36.129121 (+ 2603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:36.129143 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.130830 (+ 1687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.130836 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:36.131031 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:36.131078 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 12:21:36.131221 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 12:21:36.131421 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":18481,"mutex_wait_us":1,"thread_start_us":19668,"threads_started":1}
I20260809 12:21:36.132493 14587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:36.105854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58604 (local address 127.19.183.195:35233)
0809 12:21:36.122164 (+ 16310us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:36.122176 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:36.124206 (+ 2030us) server_negotiation.cc:407] Connection header received
0809 12:21:36.124440 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:36.124448 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:36.124608 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:36.124806 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:36.125868 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.127432 (+ 1564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.129486 (+ 2054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.130628 (+ 1142us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.130908 (+ 280us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:36.131146 (+ 238us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:36.131932 (+ 786us) server_negotiation.cc:299] Negotiation successful
0809 12:21:36.132124 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16098,"thread_start_us":15907,"threads_started":1}
I20260809 12:21:36.132736 14585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:36.105015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36594 (local address 127.19.183.193:41787)
0809 12:21:36.121552 (+ 16537us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:36.121565 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:36.123307 (+ 1742us) server_negotiation.cc:407] Connection header received
0809 12:21:36.123647 (+ 340us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:36.123655 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:36.123939 (+ 284us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:36.124137 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:36.125006 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.126337 (+ 1331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.129366 (+ 3029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.130593 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.131888 (+ 1295us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:36.132114 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:36.132182 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 12:21:36.132348 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16279,"thread_start_us":16102,"threads_started":1}
I20260809 12:21:36.133013 14582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:36.104462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:35233 (local address 127.0.0.1:58604)
0809 12:21:36.124062 (+ 19600us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:36.124081 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:36.124249 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:36.125078 (+ 829us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:36.125083 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:36.125126 (+ 43us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:36.125672 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:36.125686 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.127612 (+ 1926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.127618 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:36.129245 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:36.129259 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.131276 (+ 2017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.131282 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:36.131447 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:36.131477 (+ 30us) client_negotiation.cc:769] Sending connection context
0809 12:21:36.131560 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 12:21:36.131682 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":19458,"mutex_wait_us":26,"spinlock_wait_cycles":3840,"thread_start_us":17889,"threads_started":1}
I20260809 12:21:36.134388 14583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:36.103988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38756 (local address 127.19.183.194:46125)
0809 12:21:36.123258 (+ 19270us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:36.123269 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:36.123310 (+ 41us) server_negotiation.cc:407] Connection header received
0809 12:21:36.123454 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:36.123465 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:36.124597 (+ 1132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:36.124802 (+ 205us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:36.125824 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.127404 (+ 1580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.130143 (+ 2739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.131592 (+ 1449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.131751 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:36.131954 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:36.133858 (+ 1904us) server_negotiation.cc:299] Negotiation successful
0809 12:21:36.134034 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19056,"thread_start_us":18899,"threads_started":1}
I20260809 12:21:36.134671 14580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:36.103780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:46125 (local address 127.0.0.1:38756)
0809 12:21:36.119507 (+ 15727us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:36.119557 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:36.119765 (+ 208us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:36.125078 (+ 5313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:36.125083 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:36.125129 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:36.125637 (+ 508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:36.125651 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.127585 (+ 1934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.127591 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:36.129929 (+ 2338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:36.129946 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.132100 (+ 2154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.132106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:36.132306 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:36.132338 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 12:21:36.132425 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 12:21:36.132560 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15428,"mutex_wait_us":3,"spinlock_wait_cycles":768,"thread_start_us":15208,"threads_started":1}
I20260809 12:21:36.136544 14071 tablet_service.cc:1467] Processing CreateTablet for tablet 6c4f490ac44341b0a4bd4fe4c5584644 (DEFAULT_TABLE table=test-table [id=75d4a35d17c24bb1888c023501168914]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:21:36.136622 14473 tablet_service.cc:1467] Processing CreateTablet for tablet 6c4f490ac44341b0a4bd4fe4c5584644 (DEFAULT_TABLE table=test-table [id=75d4a35d17c24bb1888c023501168914]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:21:36.138161 14473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c4f490ac44341b0a4bd4fe4c5584644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:36.138396 14071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c4f490ac44341b0a4bd4fe4c5584644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:36.138012 14260 tablet_service.cc:1467] Processing CreateTablet for tablet 6c4f490ac44341b0a4bd4fe4c5584644 (DEFAULT_TABLE table=test-table [id=75d4a35d17c24bb1888c023501168914]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:21:36.139286 14260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6c4f490ac44341b0a4bd4fe4c5584644. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:36.149751 14589 tablet_bootstrap.cc:492] T 6c4f490ac44341b0a4bd4fe4c5584644 P 2b7cd5a49046400ca9400368c9617f7e: Bootstrap starting.
I20260809 12:21:36.153302 14592 tablet_bootstrap.cc:492] T 6c4f490ac44341b0a4bd4fe4c5584644 P 57ae388037bc4e789133ff2e9b1a35d3: Bootstrap starting.
I20260809 12:21:36.154028 14589 tablet_bootstrap.cc:654] T 6c4f490ac44341b0a4bd4fe4c5584644 P 2b7cd5a49046400ca9400368c9617f7e: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:36.155521 14589 log.cc:824] T 6c4f490ac44341b0a4bd4fe4c5584644 P 2b7cd5a49046400ca9400368c9617f7e: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:36.157287 14592 tablet_bootstrap.cc:654] T 6c4f490ac44341b0a4bd4fe4c5584644 P 57ae388037bc4e789133ff2e9b1a35d3: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:36.158548 14592 log.cc:824] T 6c4f490ac44341b0a4bd4fe4c5584644 P 57ae388037bc4e789133ff2e9b1a35d3: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:36.158847 14591 tablet_bootstrap.cc:492] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e: Bootstrap starting.
I20260809 12:21:36.158962 14589 tablet_bootstrap.cc:492] T 6c4f490ac44341b0a4bd4fe4c5584644 P 2b7cd5a49046400ca9400368c9617f7e: No bootstrap required, opened a new log
I20260809 12:21:36.159283 14589 ts_tablet_manager.cc:1397] T 6c4f490ac44341b0a4bd4fe4c5584644 P 2b7cd5a49046400ca9400368c9617f7e: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260809 12:21:36.162283 14591 tablet_bootstrap.cc:654] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:36.162309 14592 tablet_bootstrap.cc:492] T 6c4f490ac44341b0a4bd4fe4c5584644 P 57ae388037bc4e789133ff2e9b1a35d3: No bootstrap required, opened a new log
I20260809 12:21:36.162549 14592 ts_tablet_manager.cc:1397] T 6c4f490ac44341b0a4bd4fe4c5584644 P 57ae388037bc4e789133ff2e9b1a35d3: Time spent bootstrapping tablet: real 0.009s user 0.003s sys 0.004s
I20260809 12:21:36.163367 14591 log.cc:824] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:36.165863 14591 tablet_bootstrap.cc:492] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e: No bootstrap required, opened a new log
I20260809 12:21:36.166133 14591 ts_tablet_manager.cc:1397] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.005s
I20260809 12:21:36.170015 14589 raft_consensus.cc:348] T 6c4f490ac44341b0a4bd4fe4c5584644 P 2b7cd5a49046400ca9400368c9617f7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ae388037bc4e789133ff2e9b1a35d3" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 46125 } } peers { permanent_uuid: "2b7cd5a49046400ca9400368c9617f7e" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 41787 } } peers { permanent_uuid: "d97903fd90534946ae3b1aee8d90525e" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35233 } }
I20260809 12:21:36.170599 14589 raft_consensus.cc:374] T 6c4f490ac44341b0a4bd4fe4c5584644 P 2b7cd5a49046400ca9400368c9617f7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:36.170764 14589 raft_consensus.cc:729] T 6c4f490ac44341b0a4bd4fe4c5584644 P 2b7cd5a49046400ca9400368c9617f7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b7cd5a49046400ca9400368c9617f7e, State: Initialized, Role: FOLLOWER
I20260809 12:21:36.171578 14589 consensus_queue.cc:260] T 6c4f490ac44341b0a4bd4fe4c5584644 P 2b7cd5a49046400ca9400368c9617f7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ae388037bc4e789133ff2e9b1a35d3" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 46125 } } peers { permanent_uuid: "2b7cd5a49046400ca9400368c9617f7e" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 41787 } } peers { permanent_uuid: "d97903fd90534946ae3b1aee8d90525e" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35233 } }
I20260809 12:21:36.173081 14592 raft_consensus.cc:348] T 6c4f490ac44341b0a4bd4fe4c5584644 P 57ae388037bc4e789133ff2e9b1a35d3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ae388037bc4e789133ff2e9b1a35d3" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 46125 } } peers { permanent_uuid: "2b7cd5a49046400ca9400368c9617f7e" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 41787 } } peers { permanent_uuid: "d97903fd90534946ae3b1aee8d90525e" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35233 } }
I20260809 12:21:36.173529 14589 ts_tablet_manager.cc:1428] T 6c4f490ac44341b0a4bd4fe4c5584644 P 2b7cd5a49046400ca9400368c9617f7e: Time spent starting tablet: real 0.014s user 0.009s sys 0.004s
I20260809 12:21:36.173612 14592 raft_consensus.cc:374] T 6c4f490ac44341b0a4bd4fe4c5584644 P 57ae388037bc4e789133ff2e9b1a35d3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:36.173777 14592 raft_consensus.cc:729] T 6c4f490ac44341b0a4bd4fe4c5584644 P 57ae388037bc4e789133ff2e9b1a35d3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57ae388037bc4e789133ff2e9b1a35d3, State: Initialized, Role: FOLLOWER
I20260809 12:21:36.174484 14592 consensus_queue.cc:260] T 6c4f490ac44341b0a4bd4fe4c5584644 P 57ae388037bc4e789133ff2e9b1a35d3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ae388037bc4e789133ff2e9b1a35d3" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 46125 } } peers { permanent_uuid: "2b7cd5a49046400ca9400368c9617f7e" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 41787 } } peers { permanent_uuid: "d97903fd90534946ae3b1aee8d90525e" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35233 } }
I20260809 12:21:36.174481 14591 raft_consensus.cc:348] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ae388037bc4e789133ff2e9b1a35d3" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 46125 } } peers { permanent_uuid: "2b7cd5a49046400ca9400368c9617f7e" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 41787 } } peers { permanent_uuid: "d97903fd90534946ae3b1aee8d90525e" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35233 } }
I20260809 12:21:36.174968 14591 raft_consensus.cc:374] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:36.175153 14591 raft_consensus.cc:729] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d97903fd90534946ae3b1aee8d90525e, State: Initialized, Role: FOLLOWER
I20260809 12:21:36.175644 14359 heartbeater.cc:503] Master 127.19.183.254:35113 was elected leader, sending a full tablet report...
I20260809 12:21:36.175930 14591 consensus_queue.cc:260] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ae388037bc4e789133ff2e9b1a35d3" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 46125 } } peers { permanent_uuid: "2b7cd5a49046400ca9400368c9617f7e" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 41787 } } peers { permanent_uuid: "d97903fd90534946ae3b1aee8d90525e" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35233 } }
I20260809 12:21:36.177215 14561 heartbeater.cc:503] Master 127.19.183.254:35113 was elected leader, sending a full tablet report...
I20260809 12:21:36.177717 14591 ts_tablet_manager.cc:1428] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260809 12:21:36.180058 14592 ts_tablet_manager.cc:1428] T 6c4f490ac44341b0a4bd4fe4c5584644 P 57ae388037bc4e789133ff2e9b1a35d3: Time spent starting tablet: real 0.017s user 0.014s sys 0.001s
I20260809 12:21:36.239399 14599 raft_consensus.cc:482] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:21:36.239709 14599 raft_consensus.cc:504] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ae388037bc4e789133ff2e9b1a35d3" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 46125 } } peers { permanent_uuid: "2b7cd5a49046400ca9400368c9617f7e" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 41787 } } peers { permanent_uuid: "d97903fd90534946ae3b1aee8d90525e" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35233 } }
I20260809 12:21:36.241500 14599 leader_election.cc:288] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 57ae388037bc4e789133ff2e9b1a35d3 (127.19.183.194:46125), 2b7cd5a49046400ca9400368c9617f7e (127.19.183.193:41787)
I20260809 12:21:36.248587 14564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:36.241976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:46125 (local address 127.19.183.195:34863)
0809 12:21:36.242364 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:36.242387 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:36.242542 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:36.243106 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:36.243113 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:36.243143 (+ 30us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:36.243362 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:36.243375 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.244734 (+ 1359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.244738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:36.246769 (+ 2031us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:36.246784 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.248055 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.248059 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:36.248170 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:36.248193 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 12:21:36.248264 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 12:21:36.248358 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"mutex_wait_us":1}
I20260809 12:21:36.248728 14583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:36.241977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:34863 (local address 127.19.183.194:46125)
0809 12:21:36.242210 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:36.242219 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:36.242496 (+ 277us) server_negotiation.cc:407] Connection header received
0809 12:21:36.242761 (+ 265us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:36.242771 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:36.242886 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:36.242990 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:36.243558 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.244600 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.246923 (+ 2323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.247907 (+ 984us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.248058 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:36.248146 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:36.248325 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 12:21:36.248454 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 12:21:36.249622 14585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:36.242172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:57633 (local address 127.19.183.193:41787)
0809 12:21:36.242410 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:36.242419 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:36.243297 (+ 878us) server_negotiation.cc:407] Connection header received
0809 12:21:36.243502 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:36.243508 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:36.243610 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:36.243705 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:36.244296 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.245245 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.247453 (+ 2208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.248690 (+ 1237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.248883 (+ 193us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:36.248972 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:36.249306 (+ 334us) server_negotiation.cc:299] Negotiation successful
0809 12:21:36.249455 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 12:21:36.249691 14600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:36.242056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:41787 (local address 127.19.183.195:57633)
0809 12:21:36.243192 (+ 1136us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:36.243218 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:36.243353 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:36.243759 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:36.243763 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:36.243788 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:36.244142 (+ 354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:36.244153 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.245401 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.245407 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:36.247253 (+ 1846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:36.247264 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.249008 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.249014 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:36.249150 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:36.249172 (+ 22us) client_negotiation.cc:769] Sending connection context
0809 12:21:36.249243 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 12:21:36.249363 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":991,"thread_start_us":765,"threads_started":1}
I20260809 12:21:36.249908 14286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c4f490ac44341b0a4bd4fe4c5584644" candidate_uuid: "d97903fd90534946ae3b1aee8d90525e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57ae388037bc4e789133ff2e9b1a35d3" is_pre_election: true
I20260809 12:21:36.250483 14286 raft_consensus.cc:2454] T 6c4f490ac44341b0a4bd4fe4c5584644 P 57ae388037bc4e789133ff2e9b1a35d3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d97903fd90534946ae3b1aee8d90525e in term 0.
I20260809 12:21:36.250761 14107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c4f490ac44341b0a4bd4fe4c5584644" candidate_uuid: "d97903fd90534946ae3b1aee8d90525e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b7cd5a49046400ca9400368c9617f7e" is_pre_election: true
I20260809 12:21:36.251305 14393 leader_election.cc:302] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57ae388037bc4e789133ff2e9b1a35d3, d97903fd90534946ae3b1aee8d90525e; no voters:
I20260809 12:21:36.251341 14107 raft_consensus.cc:2454] T 6c4f490ac44341b0a4bd4fe4c5584644 P 2b7cd5a49046400ca9400368c9617f7e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d97903fd90534946ae3b1aee8d90525e in term 0.
I20260809 12:21:36.251695 14599 raft_consensus.cc:2781] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:21:36.251869 14599 raft_consensus.cc:482] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:21:36.251977 14599 raft_consensus.cc:3037] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:36.254213 14599 raft_consensus.cc:504] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ae388037bc4e789133ff2e9b1a35d3" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 46125 } } peers { permanent_uuid: "2b7cd5a49046400ca9400368c9617f7e" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 41787 } } peers { permanent_uuid: "d97903fd90534946ae3b1aee8d90525e" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35233 } }
I20260809 12:21:36.254848 14599 leader_election.cc:288] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e [CANDIDATE]: Term 1 election: Requested vote from peers 57ae388037bc4e789133ff2e9b1a35d3 (127.19.183.194:46125), 2b7cd5a49046400ca9400368c9617f7e (127.19.183.193:41787)
I20260809 12:21:36.255242 14286 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c4f490ac44341b0a4bd4fe4c5584644" candidate_uuid: "d97903fd90534946ae3b1aee8d90525e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "57ae388037bc4e789133ff2e9b1a35d3"
I20260809 12:21:36.255349 14107 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6c4f490ac44341b0a4bd4fe4c5584644" candidate_uuid: "d97903fd90534946ae3b1aee8d90525e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2b7cd5a49046400ca9400368c9617f7e"
I20260809 12:21:36.255460 14286 raft_consensus.cc:3037] T 6c4f490ac44341b0a4bd4fe4c5584644 P 57ae388037bc4e789133ff2e9b1a35d3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:36.255549 14107 raft_consensus.cc:3037] T 6c4f490ac44341b0a4bd4fe4c5584644 P 2b7cd5a49046400ca9400368c9617f7e [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:36.258828 14107 raft_consensus.cc:2454] T 6c4f490ac44341b0a4bd4fe4c5584644 P 2b7cd5a49046400ca9400368c9617f7e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d97903fd90534946ae3b1aee8d90525e in term 1.
I20260809 12:21:36.258829 14286 raft_consensus.cc:2454] T 6c4f490ac44341b0a4bd4fe4c5584644 P 57ae388037bc4e789133ff2e9b1a35d3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d97903fd90534946ae3b1aee8d90525e in term 1.
I20260809 12:21:36.259222 14393 leader_election.cc:302] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 57ae388037bc4e789133ff2e9b1a35d3, d97903fd90534946ae3b1aee8d90525e; no voters:
I20260809 12:21:36.259488 14599 raft_consensus.cc:2781] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:21:36.259914 14599 raft_consensus.cc:686] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e [term 1 LEADER]: Becoming Leader. State: Replica: d97903fd90534946ae3b1aee8d90525e, State: Running, Role: LEADER
I20260809 12:21:36.260480 14599 consensus_queue.cc:237] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ae388037bc4e789133ff2e9b1a35d3" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 46125 } } peers { permanent_uuid: "2b7cd5a49046400ca9400368c9617f7e" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 41787 } } peers { permanent_uuid: "d97903fd90534946ae3b1aee8d90525e" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35233 } }
I20260809 12:21:36.265044 13881 catalog_manager.cc:5495] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e reported cstate change: term changed from 0 to 1, leader changed from <none> to d97903fd90534946ae3b1aee8d90525e (127.19.183.195). New cstate: current_term: 1 leader_uuid: "d97903fd90534946ae3b1aee8d90525e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "57ae388037bc4e789133ff2e9b1a35d3" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 46125 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b7cd5a49046400ca9400368c9617f7e" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 41787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d97903fd90534946ae3b1aee8d90525e" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35233 } health_report { overall_health: HEALTHY } } }
I20260809 12:21:36.284569 14587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:36.279461 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58616 (local address 127.19.183.195:35233)
0809 12:21:36.279637 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:36.279642 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:36.279758 (+ 116us) server_negotiation.cc:407] Connection header received
0809 12:21:36.279972 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:36.279975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:36.280047 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:36.280169 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:21:36.280491 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.281204 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.282609 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.283064 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.283168 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:36.283634 (+ 466us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:21:36.284129 (+ 495us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:21:36.284238 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:36.284310 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 12:21:36.284370 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 12:21:36.292532 14286 raft_consensus.cc:1261] T 6c4f490ac44341b0a4bd4fe4c5584644 P 57ae388037bc4e789133ff2e9b1a35d3 [term 1 FOLLOWER]: Refusing update from remote peer d97903fd90534946ae3b1aee8d90525e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:21:36.292534 14107 raft_consensus.cc:1261] T 6c4f490ac44341b0a4bd4fe4c5584644 P 2b7cd5a49046400ca9400368c9617f7e [term 1 FOLLOWER]: Refusing update from remote peer d97903fd90534946ae3b1aee8d90525e: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:21:36.293447 14601 consensus_queue.cc:1035] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e [LEADER]: Connected to new peer: Peer: permanent_uuid: "57ae388037bc4e789133ff2e9b1a35d3" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 46125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:21:36.293854 14599 consensus_queue.cc:1035] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e [LEADER]: Connected to new peer: Peer: permanent_uuid: "2b7cd5a49046400ca9400368c9617f7e" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 41787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 12:21:36.296707 14563 tablet.cc:2369] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:21:36.313937 14604 mvcc.cc:205] Tried to move back new op lower bound from 7316595082404298752 to 7316595082287075328. Current Snapshot: MvccSnapshot[applied={T|T < 7316595082404298752 or (T in {7316595082404298752})}]
I20260809 12:21:36.320596 13881 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:50692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:21:36.320900 13881 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:50692:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:21:36.324687 13881 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5234cf93dc664bda819e10c4d261cc00: Sending DeleteTablet for 3 replicas of tablet 6c4f490ac44341b0a4bd4fe4c5584644
I20260809 12:21:36.325775 14260 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c4f490ac44341b0a4bd4fe4c5584644 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:21:36 UTC) from {username='slave'} at 127.0.0.1:38756
I20260809 12:21:36.326249 14473 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c4f490ac44341b0a4bd4fe4c5584644 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:21:36 UTC) from {username='slave'} at 127.0.0.1:58604
I20260809 12:21:36.326843 14071 tablet_service.cc:1514] Processing DeleteTablet for tablet 6c4f490ac44341b0a4bd4fe4c5584644 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:21:36 UTC) from {username='slave'} at 127.0.0.1:36594
I20260809 12:21:36.326691 14618 tablet_replica.cc:330] T 6c4f490ac44341b0a4bd4fe4c5584644 P 57ae388037bc4e789133ff2e9b1a35d3: stopping tablet replica
I20260809 12:21:36.328702 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 13977
I20260809 12:21:36.328642 14618 raft_consensus.cc:2229] T 6c4f490ac44341b0a4bd4fe4c5584644 P 57ae388037bc4e789133ff2e9b1a35d3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:21:36.329319 14618 raft_consensus.cc:2258] T 6c4f490ac44341b0a4bd4fe4c5584644 P 57ae388037bc4e789133ff2e9b1a35d3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:21:36.331911 14618 ts_tablet_manager.cc:1905] T 6c4f490ac44341b0a4bd4fe4c5584644 P 57ae388037bc4e789133ff2e9b1a35d3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:21:36.336829 14622 tablet_replica.cc:330] T 6c4f490ac44341b0a4bd4fe4c5584644 P 2b7cd5a49046400ca9400368c9617f7e: stopping tablet replica
I20260809 12:21:36.337375 14622 raft_consensus.cc:2229] T 6c4f490ac44341b0a4bd4fe4c5584644 P 2b7cd5a49046400ca9400368c9617f7e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:21:36.337903 14622 raft_consensus.cc:2258] T 6c4f490ac44341b0a4bd4fe4c5584644 P 2b7cd5a49046400ca9400368c9617f7e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:21:36.340147 14618 ts_tablet_manager.cc:1918] T 6c4f490ac44341b0a4bd4fe4c5584644 P 57ae388037bc4e789133ff2e9b1a35d3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:21:36.340287 14622 ts_tablet_manager.cc:1905] T 6c4f490ac44341b0a4bd4fe4c5584644 P 2b7cd5a49046400ca9400368c9617f7e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:21:36.340379 14618 log.cc:1196] T 6c4f490ac44341b0a4bd4fe4c5584644 P 57ae388037bc4e789133ff2e9b1a35d3: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/6c4f490ac44341b0a4bd4fe4c5584644
I20260809 12:21:36.341029 14618 ts_tablet_manager.cc:1939] T 6c4f490ac44341b0a4bd4fe4c5584644 P 57ae388037bc4e789133ff2e9b1a35d3: Deleting consensus metadata
I20260809 12:21:36.342790 13864 catalog_manager.cc:4841] TS 57ae388037bc4e789133ff2e9b1a35d3 (127.19.183.194:46125): tablet 6c4f490ac44341b0a4bd4fe4c5584644 (table test-table [id=75d4a35d17c24bb1888c023501168914]) successfully deleted
I20260809 12:21:36.346467 14622 ts_tablet_manager.cc:1918] T 6c4f490ac44341b0a4bd4fe4c5584644 P 2b7cd5a49046400ca9400368c9617f7e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:21:36.338667 14621 tablet_replica.cc:330] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e: stopping tablet replica
I20260809 12:21:36.347039 14621 raft_consensus.cc:2229] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:21:36.347731 14621 raft_consensus.cc:2258] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:21:36.348138 14622 log.cc:1196] T 6c4f490ac44341b0a4bd4fe4c5584644 P 2b7cd5a49046400ca9400368c9617f7e: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/6c4f490ac44341b0a4bd4fe4c5584644
I20260809 12:21:36.348686 14622 ts_tablet_manager.cc:1939] T 6c4f490ac44341b0a4bd4fe4c5584644 P 2b7cd5a49046400ca9400368c9617f7e: Deleting consensus metadata
I20260809 12:21:36.350448 14621 ts_tablet_manager.cc:1905] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:21:36.350454 13865 catalog_manager.cc:4841] TS 2b7cd5a49046400ca9400368c9617f7e (127.19.183.193:41787): tablet 6c4f490ac44341b0a4bd4fe4c5584644 (table test-table [id=75d4a35d17c24bb1888c023501168914]) successfully deleted
I20260809 12:21:36.352516 14585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:36.330259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36608 (local address 127.19.183.193:41787)
0809 12:21:36.330445 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:36.330451 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:36.343952 (+ 13501us) server_negotiation.cc:407] Connection header received
0809 12:21:36.344164 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:36.344182 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:36.344294 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:36.344426 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:36.345186 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.346414 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.348410 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.349109 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.349219 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:36.351043 (+ 1824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:36.351051 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:36.351063 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:36.351597 (+ 534us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:36.351640 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:36.351658 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:36.351795 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:36.352010 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 12:21:36.352147 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 12:21:36.353201 14158 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:36.358527 14621 ts_tablet_manager.cc:1918] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:21:36.358745 14621 log.cc:1196] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/6c4f490ac44341b0a4bd4fe4c5584644
I20260809 12:21:36.359357 14621 ts_tablet_manager.cc:1939] T 6c4f490ac44341b0a4bd4fe4c5584644 P d97903fd90534946ae3b1aee8d90525e: Deleting consensus metadata
I20260809 12:21:36.361114 13862 catalog_manager.cc:4841] TS d97903fd90534946ae3b1aee8d90525e (127.19.183.195:35233): tablet 6c4f490ac44341b0a4bd4fe4c5584644 (table test-table [id=75d4a35d17c24bb1888c023501168914]) successfully deleted
I20260809 12:21:36.442461 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 13977
I20260809 12:21:36.472919 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 14174
I20260809 12:21:36.480494 14583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:36.473810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38762 (local address 127.19.183.194:46125)
0809 12:21:36.474016 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:36.474022 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:36.474043 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:21:36.474232 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:36.474238 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:36.474368 (+ 130us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:36.474487 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:36.474972 (+ 485us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.475695 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.477152 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.477551 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.477693 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:36.479029 (+ 1336us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:36.479038 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:36.479053 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:36.479702 (+ 649us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:36.479763 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:36.479784 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:36.479904 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:36.480044 (+ 140us) server_negotiation.cc:299] Negotiation successful
0809 12:21:36.480159 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 12:21:36.481190 14352 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:36.598456 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 14174
I20260809 12:21:36.623669 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 14365
I20260809 12:21:36.631309 14587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:36.624606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58630 (local address 127.19.183.195:35233)
0809 12:21:36.624781 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:36.624789 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:36.624846 (+ 57us) server_negotiation.cc:407] Connection header received
0809 12:21:36.625082 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:36.625092 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:36.625192 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:36.625327 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:36.625776 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.626782 (+ 1006us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.628141 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:36.628551 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:36.628701 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:36.629854 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:36.629862 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:36.629880 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:36.630497 (+ 617us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:36.630543 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:36.630565 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:36.630700 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:36.630868 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 12:21:36.630970 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 12:21:36.632126 14554 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:36.752558 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 14365
I20260809 12:21:36.782218 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 13839
I20260809 12:21:36.783434 13929 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:36.891193 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 13839
2026-08-09T12:21:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (2955 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:21:37 dist-test-slave-39l8 krb5kdc[14632](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:21:37 dist-test-slave-39l8 krb5kdc[14632](info): set up 2 sockets
Aug 09 12:21:37 dist-test-slave-39l8 krb5kdc[14632](info): commencing operation
krb5kdc: starting...
W20260809 12:21:38.201877 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.263s user 0.003s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:21:38 dist-test-slave-39l8 krb5kdc[14632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:21:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:21:38Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:21:38.515405 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:42639
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:42639
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:21:38.871172 14653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:38.871454 14653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:38.871553 14653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:38.878850 14653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:21:38.878957 14653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:38.879004 14653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:38.879063 14653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:21:38.879106 14653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:21:38.888953 14653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:42639
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:42639
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:38.890172 14653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:38.891461 14653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:38.903268 14663 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:38.903355 14662 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:38.903374 14665 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:38.904464 14653 server_base.cc:1034] running on GCE node
I20260809 12:21:38.905156 14653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:38.906565 14653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:38.907768 14653 hybrid_clock.cc:648] HybridClock initialized: now 1786278098907704 us; error 57 us; skew 500 ppm
I20260809 12:21:38.908115 14653 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:21:38 dist-test-slave-39l8 krb5kdc[14632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278098, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:21:38.911223 14653 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:21:38.912694 14653 webserver.cc:460] Webserver started at http://127.19.183.254:38527/ using document root <none> and password file <none>
I20260809 12:21:38.913350 14653 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:38.913458 14653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:38.913697 14653 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:38.916741 14653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "d7e4550d23cf40aba2f67525cc24f2de"
format_stamp: "Formatted at 2026-08-09 12:21:38 on dist-test-slave-39l8"
server_key: "0195f9a4e34120095f32d6045b859d5b"
server_key_iv: "b181bc96c8911b0a4c14565e655fb6f6"
server_key_version: "encryptionkey@0"
I20260809 12:21:38.917477 14653 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "d7e4550d23cf40aba2f67525cc24f2de"
format_stamp: "Formatted at 2026-08-09 12:21:38 on dist-test-slave-39l8"
server_key: "0195f9a4e34120095f32d6045b859d5b"
server_key_iv: "b181bc96c8911b0a4c14565e655fb6f6"
server_key_version: "encryptionkey@0"
I20260809 12:21:38.922706 14653 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 12:21:38.926129 14673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:38.927501 14653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:21:38.927690 14653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "d7e4550d23cf40aba2f67525cc24f2de"
format_stamp: "Formatted at 2026-08-09 12:21:38 on dist-test-slave-39l8"
server_key: "0195f9a4e34120095f32d6045b859d5b"
server_key_iv: "b181bc96c8911b0a4c14565e655fb6f6"
server_key_version: "encryptionkey@0"
I20260809 12:21:38.927891 14653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:38.942811 14653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:38.946692 14653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:38.947041 14653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:38.965544 14653 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:42639
I20260809 12:21:38.965564 14746 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:42639 every 8 connection(s)
I20260809 12:21:38.967011 14653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:21:38.967749 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 14653
I20260809 12:21:38.967942 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:21:38.968497 20191 external_mini_cluster.cc:1428] Setting key 2bbfd38ec96b0a237518fc2e71afb771
I20260809 12:21:38.971604 14749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 12:21:38 dist-test-slave-39l8 krb5kdc[14632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:21:38.984870 14749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de: Bootstrap starting.
I20260809 12:21:38.988732 14749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:38.990048 14749 log.cc:824] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:38.992877 14755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:38.970399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51282 (local address 127.19.183.254:42639)
0809 12:21:38.971175 (+ 776us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:38.971186 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:38.971230 (+ 44us) server_negotiation.cc:407] Connection header received
0809 12:21:38.972891 (+ 1661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:38.972913 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:38.973256 (+ 343us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:38.973577 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:38.975463 (+ 1886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:38.976806 (+ 1343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:38.978810 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:38.979516 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:38.979644 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:38.984504 (+ 4860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:38.984515 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:38.984527 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:38.984566 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:38.988382 (+ 3816us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:38.989095 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:38.989105 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:38.989115 (+ 10us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:38.989349 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:38.989729 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:38.989733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:38.989736 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:38.990441 (+ 705us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:38.990845 (+ 404us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:38.991232 (+ 387us) server_negotiation.cc:299] Negotiation successful
0809 12:21:38.991589 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":377,"threads_started":1}
I20260809 12:21:38.993510 14749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de: No bootstrap required, opened a new log
I20260809 12:21:39.001252 14749 raft_consensus.cc:348] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e4550d23cf40aba2f67525cc24f2de" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 42639 } }
I20260809 12:21:39.001566 14749 raft_consensus.cc:374] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:39.001643 14749 raft_consensus.cc:729] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d7e4550d23cf40aba2f67525cc24f2de, State: Initialized, Role: FOLLOWER
I20260809 12:21:39.002167 14749 consensus_queue.cc:260] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e4550d23cf40aba2f67525cc24f2de" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 42639 } }
I20260809 12:21:39.002372 14749 raft_consensus.cc:388] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:21:39.002467 14749 raft_consensus.cc:482] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:21:39.002596 14749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:39.004691 14749 raft_consensus.cc:504] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e4550d23cf40aba2f67525cc24f2de" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 42639 } }
I20260809 12:21:39.005122 14749 leader_election.cc:302] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d7e4550d23cf40aba2f67525cc24f2de; no voters:
I20260809 12:21:39.005681 14749 leader_election.cc:288] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:21:39.005822 14757 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:21:39.007099 14757 raft_consensus.cc:686] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de [term 1 LEADER]: Becoming Leader. State: Replica: d7e4550d23cf40aba2f67525cc24f2de, State: Running, Role: LEADER
I20260809 12:21:39.007625 14757 consensus_queue.cc:237] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e4550d23cf40aba2f67525cc24f2de" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 42639 } }
I20260809 12:21:39.007884 14749 sys_catalog.cc:564] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:21:39.010394 14760 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de [sys.catalog]: SysCatalogTable state changed. Reason: New leader d7e4550d23cf40aba2f67525cc24f2de. Latest consensus state: current_term: 1 leader_uuid: "d7e4550d23cf40aba2f67525cc24f2de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e4550d23cf40aba2f67525cc24f2de" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 42639 } } }
I20260809 12:21:39.010848 14760 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de [sys.catalog]: This master's current role is: LEADER
I20260809 12:21:39.010406 14759 sys_catalog.cc:455] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d7e4550d23cf40aba2f67525cc24f2de" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d7e4550d23cf40aba2f67525cc24f2de" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 42639 } } }
I20260809 12:21:39.011363 14759 sys_catalog.cc:458] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de [sys.catalog]: This master's current role is: LEADER
I20260809 12:21:39.013774 14765 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:21:39.019029 14765 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:21:39.027805 14765 catalog_manager.cc:1347] Generated new cluster ID: 4e33e531327b4d198965e8964548e338
I20260809 12:21:39.027925 14765 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:21:39.048981 14765 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:21:39.050963 14765 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:21:39.079933 14765 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de: Generated new TSK 0
I20260809 12:21:39.081019 14765 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
I20260809 12:21:39.156208 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:42639
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:21:39.505561 14788 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:39.505839 14788 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:39.505997 14788 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:39.513082 14788 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:39.513230 14788 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:39.513393 14788 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:21:39.523180 14788 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:42639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:39.524435 14788 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:39.525784 14788 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:39.538795 14800 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:39.538884 14797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:39.538905 14795 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:39.539927 14788 server_base.cc:1034] running on GCE node
I20260809 12:21:39.540531 14788 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:39.541369 14788 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:39.542567 14788 hybrid_clock.cc:648] HybridClock initialized: now 1786278099542529 us; error 26 us; skew 500 ppm
I20260809 12:21:39.542900 14788 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:21:39 dist-test-slave-39l8 krb5kdc[14632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278099, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:21:39.545881 14788 init.cc:377] Logged in from keytab as kudu/127.19.183.193@KRBTEST.COM (short username kudu)
I20260809 12:21:39.547318 14788 webserver.cc:460] Webserver started at http://127.19.183.193:35479/ using document root <none> and password file <none>
I20260809 12:21:39.547979 14788 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:39.548131 14788 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:39.548446 14788 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:39.551330 14788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "822633ef5f694d8580916a4f0c5ee400"
format_stamp: "Formatted at 2026-08-09 12:21:39 on dist-test-slave-39l8"
server_key: "1d4d6dfbd2ba90eab84cf6e70e426c99"
server_key_iv: "ea17f008a0c7d771c4962a05ff601fe6"
server_key_version: "encryptionkey@0"
I20260809 12:21:39.552141 14788 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "822633ef5f694d8580916a4f0c5ee400"
format_stamp: "Formatted at 2026-08-09 12:21:39 on dist-test-slave-39l8"
server_key: "1d4d6dfbd2ba90eab84cf6e70e426c99"
server_key_iv: "ea17f008a0c7d771c4962a05ff601fe6"
server_key_version: "encryptionkey@0"
I20260809 12:21:39.557374 14788 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:21:39.560719 14810 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:39.562029 14788 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:21:39.562237 14788 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "822633ef5f694d8580916a4f0c5ee400"
format_stamp: "Formatted at 2026-08-09 12:21:39 on dist-test-slave-39l8"
server_key: "1d4d6dfbd2ba90eab84cf6e70e426c99"
server_key_iv: "ea17f008a0c7d771c4962a05ff601fe6"
server_key_version: "encryptionkey@0"
I20260809 12:21:39.562433 14788 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:39.589388 14788 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:39.593153 14788 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:39.593444 14788 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:39.594391 14788 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:21:39.595829 14788 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:21:39.595937 14788 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:39.596086 14788 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:21:39.596168 14788 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:39.630379 14788 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:35715
I20260809 12:21:39.630450 14981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:35715 every 8 connection(s)
I20260809 12:21:39.631915 14788 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:21:39.640084 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 14788
I20260809 12:21:39.640280 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:21:39.640805 20191 external_mini_cluster.cc:1428] Setting key 376747d1f890bac09266dccd246846b3
Aug 09 12:21:39 dist-test-slave-39l8 krb5kdc[14632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278099, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:21:39.653605 14988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:39.634634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:36475 (local address 127.19.183.254:42639)
0809 12:21:39.635123 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:39.635129 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:39.636039 (+ 910us) server_negotiation.cc:407] Connection header received
0809 12:21:39.637964 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:39.637970 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:39.638067 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:39.638200 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:39.639706 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:39.640808 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:39.642524 (+ 1716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:39.643066 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:39.643212 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:39.646677 (+ 3465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:39.646685 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:39.646693 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:39.646759 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:39.650233 (+ 3474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:39.650985 (+ 752us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:39.650993 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:39.650999 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:39.651178 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:39.651620 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:39.651626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:39.651632 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:39.652052 (+ 420us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:39.652223 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:39.652879 (+ 656us) server_negotiation.cc:299] Negotiation successful
0809 12:21:39.653144 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":381,"thread_start_us":268,"threads_started":1}
I20260809 12:21:39.654211 14990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:39.635204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:42639 (local address 127.19.183.193:36475)
0809 12:21:39.635878 (+ 674us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:39.635927 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:39.637688 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:39.638434 (+ 746us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:39.638441 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:39.638988 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:21:39.639531 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:39.639548 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:39.640985 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:39.640989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:39.642357 (+ 1368us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:39.642371 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:39.643282 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:39.643288 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:39.643477 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:39.644182 (+ 705us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:21:39.644204 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:21:39.646460 (+ 2256us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:21:39.650367 (+ 3907us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:21:39.650374 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:21:39.650384 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:21:39.650873 (+ 489us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:21:39.651288 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:21:39.651291 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:21:39.651293 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:21:39.651473 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:21:39.652178 (+ 705us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:21:39.652184 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:21:39.652488 (+ 304us) client_negotiation.cc:769] Sending connection context
0809 12:21:39.652700 (+ 212us) client_negotiation.cc:240] Negotiation successful
0809 12:21:39.653067 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"thread_start_us":266,"threads_started":1}
I20260809 12:21:39.655900 14983 heartbeater.cc:347] Connected to a master server at 127.19.183.254:42639
I20260809 12:21:39.656208 14983 heartbeater.cc:464] Registering TS with master...
I20260809 12:21:39.656890 14983 heartbeater.cc:511] Master 127.19.183.254:42639 requested a full tablet report, sending...
I20260809 12:21:39.659022 14698 ts_manager.cc:194] Registered new tserver with Master: 822633ef5f694d8580916a4f0c5ee400 (127.19.183.193:35715)
I20260809 12:21:39.660331 14698 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:36475
WARNING: no policy specified for kudu/127.19.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
I20260809 12:21:39.713804 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:42639
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:21:40.070116 14994 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:40.070452 14994 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:40.070632 14994 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:40.078033 14994 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:40.078182 14994 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:40.078389 14994 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:21:40.088512 14994 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:42639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:40.089814 14994 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:40.091223 14994 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:40.104364 15001 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:40.104419 15004 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:40.104578 15002 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:40.105968 14994 server_base.cc:1034] running on GCE node
I20260809 12:21:40.106600 14994 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:40.107573 14994 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:40.108780 14994 hybrid_clock.cc:648] HybridClock initialized: now 1786278100108718 us; error 49 us; skew 500 ppm
I20260809 12:21:40.109117 14994 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:21:40 dist-test-slave-39l8 krb5kdc[14632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278100, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:21:40.112272 14994 init.cc:377] Logged in from keytab as kudu/127.19.183.194@KRBTEST.COM (short username kudu)
I20260809 12:21:40.113677 14994 webserver.cc:460] Webserver started at http://127.19.183.194:37417/ using document root <none> and password file <none>
I20260809 12:21:40.114400 14994 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:40.114552 14994 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:40.114933 14994 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:40.117839 14994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "280ec55393c142aa9b571aa095b0ed13"
format_stamp: "Formatted at 2026-08-09 12:21:40 on dist-test-slave-39l8"
server_key: "4ef7bf33c0e800896e8fc9459c02ec50"
server_key_iv: "4caea2b760cb9f5b56332baf13a0f706"
server_key_version: "encryptionkey@0"
I20260809 12:21:40.118608 14994 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "280ec55393c142aa9b571aa095b0ed13"
format_stamp: "Formatted at 2026-08-09 12:21:40 on dist-test-slave-39l8"
server_key: "4ef7bf33c0e800896e8fc9459c02ec50"
server_key_iv: "4caea2b760cb9f5b56332baf13a0f706"
server_key_version: "encryptionkey@0"
I20260809 12:21:40.123764 14994 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:21:40.127086 15016 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:40.128330 14994 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:21:40.128518 14994 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "280ec55393c142aa9b571aa095b0ed13"
format_stamp: "Formatted at 2026-08-09 12:21:40 on dist-test-slave-39l8"
server_key: "4ef7bf33c0e800896e8fc9459c02ec50"
server_key_iv: "4caea2b760cb9f5b56332baf13a0f706"
server_key_version: "encryptionkey@0"
I20260809 12:21:40.128729 14994 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:40.153084 14994 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:40.157276 14994 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:40.157562 14994 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:40.158507 14994 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:21:40.159861 14994 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:21:40.159967 14994 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:40.160110 14994 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:21:40.160202 14994 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:40.194228 14994 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:39719
I20260809 12:21:40.194276 15186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:39719 every 8 connection(s)
I20260809 12:21:40.195557 14994 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:21:40.196669 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 14994
I20260809 12:21:40.196873 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:21:40.197423 20191 external_mini_cluster.cc:1428] Setting key 64dd9519eac22aa344a5e36fb628c67a
Aug 09 12:21:40 dist-test-slave-39l8 krb5kdc[14632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278100, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:21:40.218504 15190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:40.198466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:41769 (local address 127.19.183.254:42639)
0809 12:21:40.198989 (+ 523us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:40.198995 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:40.200094 (+ 1099us) server_negotiation.cc:407] Connection header received
0809 12:21:40.202059 (+ 1965us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:40.202067 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:40.202165 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:40.202309 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:40.203824 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.204972 (+ 1148us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.206491 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.207268 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.207416 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:40.211433 (+ 4017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:40.211442 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:40.211449 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:40.211495 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:40.214997 (+ 3502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:40.216064 (+ 1067us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:40.216072 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:40.216078 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:40.216272 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:40.216816 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:40.216820 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:40.216822 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:40.217078 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:40.217205 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:40.218063 (+ 858us) server_negotiation.cc:299] Negotiation successful
0809 12:21:40.218233 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":406,"thread_start_us":284,"threads_started":1}
I20260809 12:21:40.220146 15191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:40.199064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:42639 (local address 127.19.183.194:41769)
0809 12:21:40.199925 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:40.199969 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:40.201794 (+ 1825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:40.202506 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:40.202514 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:40.203086 (+ 572us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:21:40.203649 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:40.203667 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.205138 (+ 1471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.205142 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:40.206345 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:40.206358 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.207521 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.207529 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:40.207801 (+ 272us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:40.208532 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:21:40.208603 (+ 71us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:21:40.211201 (+ 2598us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:21:40.215188 (+ 3987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:21:40.215199 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:21:40.215218 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:21:40.215906 (+ 688us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:21:40.216417 (+ 511us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:21:40.216424 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:21:40.216430 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:21:40.216692 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:21:40.217224 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:21:40.217235 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:21:40.217681 (+ 446us) client_negotiation.cc:769] Sending connection context
0809 12:21:40.217974 (+ 293us) client_negotiation.cc:240] Negotiation successful
0809 12:21:40.218555 (+ 581us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":582,"thread_start_us":365,"threads_started":1}
I20260809 12:21:40.221663 15187 heartbeater.cc:347] Connected to a master server at 127.19.183.254:42639
I20260809 12:21:40.221974 15187 heartbeater.cc:464] Registering TS with master...
I20260809 12:21:40.222666 15187 heartbeater.cc:511] Master 127.19.183.254:42639 requested a full tablet report, sending...
I20260809 12:21:40.224201 14698 ts_manager.cc:194] Registered new tserver with Master: 280ec55393c142aa9b571aa095b0ed13 (127.19.183.194:39719)
I20260809 12:21:40.225008 14698 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:41769
WARNING: no policy specified for kudu/127.19.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
I20260809 12:21:40.272289 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:42639
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:46555
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:21:40.617789 15196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:40.618119 15196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:40.618302 15196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:40.625540 15196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:40.625690 15196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:40.625869 15196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:21:40.636408 15196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:46555
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:42639
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:40.637641 15196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:40.639062 15196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:40.652618 15205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:40.652529 15206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:40.652585 15208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:40.654122 15196 server_base.cc:1034] running on GCE node
I20260809 12:21:40.654671 15196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:40.655637 15196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:40.656881 15196 hybrid_clock.cc:648] HybridClock initialized: now 1786278100656838 us; error 36 us; skew 500 ppm
I20260809 12:21:40.657326 15196 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:21:40 dist-test-slave-39l8 krb5kdc[14632](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278100, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:21:40.660828 15196 init.cc:377] Logged in from keytab as kudu/127.19.183.195@KRBTEST.COM (short username kudu)
I20260809 12:21:40.662358 15196 webserver.cc:460] Webserver started at http://127.19.183.195:42969/ using document root <none> and password file <none>
I20260809 12:21:40.662976 14983 heartbeater.cc:503] Master 127.19.183.254:42639 was elected leader, sending a full tablet report...
I20260809 12:21:40.663199 15196 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:40.663383 15196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:40.663776 15196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:40.666908 15196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "26a12e60cc8547759a9c0ecb00261590"
format_stamp: "Formatted at 2026-08-09 12:21:40 on dist-test-slave-39l8"
server_key: "e2278daeb9a1fbe8283ad813696a14e4"
server_key_iv: "0c0dc0b2c84c3c58e5bc66353c78d5b0"
server_key_version: "encryptionkey@0"
I20260809 12:21:40.667693 15196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "26a12e60cc8547759a9c0ecb00261590"
format_stamp: "Formatted at 2026-08-09 12:21:40 on dist-test-slave-39l8"
server_key: "e2278daeb9a1fbe8283ad813696a14e4"
server_key_iv: "0c0dc0b2c84c3c58e5bc66353c78d5b0"
server_key_version: "encryptionkey@0"
I20260809 12:21:40.672806 15196 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 12:21:40.676311 15222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:40.677595 15196 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:21:40.677780 15196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "26a12e60cc8547759a9c0ecb00261590"
format_stamp: "Formatted at 2026-08-09 12:21:40 on dist-test-slave-39l8"
server_key: "e2278daeb9a1fbe8283ad813696a14e4"
server_key_iv: "0c0dc0b2c84c3c58e5bc66353c78d5b0"
server_key_version: "encryptionkey@0"
I20260809 12:21:40.677981 15196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:40.698648 15196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:40.703222 15196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:40.703504 15196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:40.704486 15196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:21:40.705786 15196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:21:40.705896 15196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:40.706036 15196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:21:40.706116 15196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:40.740829 15196 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:38369
I20260809 12:21:40.740877 15401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:38369 every 8 connection(s)
I20260809 12:21:40.742090 15196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:21:40.744066 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 15196
I20260809 12:21:40.744287 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:21:40.744815 20191 external_mini_cluster.cc:1428] Setting key c80da784938bd1c20210f23943403ece
Aug 09 12:21:40 dist-test-slave-39l8 krb5kdc[14632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278100, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:21:40.761796 15406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:40.744661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:54437 (local address 127.19.183.254:42639)
0809 12:21:40.745206 (+ 545us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:40.745212 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:40.746272 (+ 1060us) server_negotiation.cc:407] Connection header received
0809 12:21:40.748499 (+ 2227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:40.748503 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:40.748587 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:40.748684 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:40.750238 (+ 1554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.751054 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.752524 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.752904 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.752994 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:40.756367 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:40.756371 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:40.756374 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:40.756404 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:40.759082 (+ 2678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:40.759920 (+ 838us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:40.759925 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:40.759928 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:40.760067 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:40.760445 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:40.760448 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:40.760450 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:40.760689 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:40.760795 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:40.761378 (+ 583us) server_negotiation.cc:299] Negotiation successful
0809 12:21:40.761526 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":253,"threads_started":1}
I20260809 12:21:40.762702 15410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:40.745273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:42639 (local address 127.19.183.195:54437)
0809 12:21:40.746103 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:40.746167 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:40.748256 (+ 2089us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:40.748965 (+ 709us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:40.748976 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:40.749528 (+ 552us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:21:40.750081 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:40.750098 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.751192 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.751196 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:40.752400 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:40.752412 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.753086 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.753091 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:40.753298 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:40.754006 (+ 708us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:21:40.754033 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:21:40.756189 (+ 2156us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:21:40.759240 (+ 3051us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:21:40.759246 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:21:40.759256 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:21:40.759805 (+ 549us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:21:40.760180 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:21:40.760184 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:21:40.760186 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:21:40.760350 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:21:40.760791 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:21:40.760797 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:21:40.761068 (+ 271us) client_negotiation.cc:769] Sending connection context
0809 12:21:40.761292 (+ 224us) client_negotiation.cc:240] Negotiation successful
0809 12:21:40.761630 (+ 338us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":317,"threads_started":1}
I20260809 12:21:40.764140 15404 heartbeater.cc:347] Connected to a master server at 127.19.183.254:42639
I20260809 12:21:40.764437 15404 heartbeater.cc:464] Registering TS with master...
I20260809 12:21:40.765126 15404 heartbeater.cc:511] Master 127.19.183.254:42639 requested a full tablet report, sending...
I20260809 12:21:40.766618 14698 ts_manager.cc:194] Registered new tserver with Master: 26a12e60cc8547759a9c0ecb00261590 (127.19.183.195:38369)
I20260809 12:21:40.767437 14698 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:54437
I20260809 12:21:40.771965 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:21:40.789852 15406 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:40.776022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51290 (local address 127.19.183.254:42639)
0809 12:21:40.776175 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:40.776180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:40.776789 (+ 609us) server_negotiation.cc:407] Connection header received
0809 12:21:40.776868 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:40.776871 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:40.776939 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:40.777018 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:40.778519 (+ 1501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.779284 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.780645 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.781011 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.781108 (+ 97us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:40.784705 (+ 3597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:40.784714 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:40.784720 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:40.784763 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:40.787312 (+ 2549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:40.788029 (+ 717us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:40.788033 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:40.788035 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:40.788153 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:40.788524 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:40.788527 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:40.788528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:40.788793 (+ 265us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:40.788980 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:40.789302 (+ 322us) server_negotiation.cc:299] Negotiation successful
0809 12:21:40.789510 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 12:21:40.793460 14698 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51290:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:21:40.800302 14698 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:21:40.837024 15430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:40.813123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58086 (local address 127.19.183.194:39719)
0809 12:21:40.825526 (+ 12403us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:40.825540 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:40.827278 (+ 1738us) server_negotiation.cc:407] Connection header received
0809 12:21:40.827555 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:40.827562 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:40.827777 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:40.827936 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:40.828863 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.830478 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.834126 (+ 3648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.835564 (+ 1438us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.835728 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:40.835872 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:40.836490 (+ 618us) server_negotiation.cc:299] Negotiation successful
0809 12:21:40.836667 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12158,"thread_start_us":12409,"threads_started":1}
I20260809 12:21:40.837252 15428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:40.812365 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:38369 (local address 127.0.0.1:33556)
0809 12:21:40.826667 (+ 14302us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:40.826760 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:40.827004 (+ 244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:40.827871 (+ 867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:40.827881 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:40.827933 (+ 52us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:40.828547 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:40.828565 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.830670 (+ 2105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.830674 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:40.833744 (+ 3070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:40.833756 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.835907 (+ 2151us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.835914 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:40.836184 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:40.836245 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 12:21:40.836407 (+ 162us) client_negotiation.cc:240] Negotiation successful
0809 12:21:40.836688 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14096,"mutex_wait_us":158,"thread_start_us":14505,"threads_started":1}
I20260809 12:21:40.837565 15426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:40.812780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:39719 (local address 127.0.0.1:58086)
0809 12:21:40.827166 (+ 14386us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:40.827184 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:40.827378 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:40.828231 (+ 853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:40.828235 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:40.828256 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:40.828633 (+ 377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:40.828641 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.830660 (+ 2019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.830665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:40.833947 (+ 3282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:40.833964 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.836062 (+ 2098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.836070 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:40.836238 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:40.836261 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 12:21:40.836407 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 12:21:40.836688 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14255,"mutex_wait_us":859,"thread_start_us":13548,"threads_started":1}
I20260809 12:21:40.837947 15427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:40.812517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33556 (local address 127.19.183.195:38369)
0809 12:21:40.826325 (+ 13808us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:40.826336 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:40.827244 (+ 908us) server_negotiation.cc:407] Connection header received
0809 12:21:40.827363 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:40.827371 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:40.827628 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:40.827828 (+ 200us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:40.828795 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.830478 (+ 1683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.833893 (+ 3415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.835374 (+ 1481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.835572 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:40.835764 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:40.837416 (+ 1652us) server_negotiation.cc:299] Negotiation successful
0809 12:21:40.837595 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13624,"thread_start_us":13449,"threads_started":1}
I20260809 12:21:40.840255 15432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:40.830243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:35715 (local address 127.0.0.1:59712)
0809 12:21:40.832255 (+ 2012us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:40.832274 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:40.832421 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:40.833128 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:40.833133 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:40.833158 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:40.833616 (+ 458us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:40.833629 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.835483 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.835489 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:40.837588 (+ 2099us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:40.837611 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.839462 (+ 1851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.839468 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:40.839622 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:40.839653 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 12:21:40.839788 (+ 135us) client_negotiation.cc:240] Negotiation successful
0809 12:21:40.839912 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1900,"thread_start_us":1786,"threads_started":1}
I20260809 12:21:40.840288 15431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:40.827675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59712 (local address 127.19.183.193:35715)
0809 12:21:40.831863 (+ 4188us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:40.831871 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:40.832387 (+ 516us) server_negotiation.cc:407] Connection header received
0809 12:21:40.832588 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:40.832605 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:40.832861 (+ 256us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:40.833118 (+ 257us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:40.833802 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.835069 (+ 1267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.837782 (+ 2713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.839259 (+ 1477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.839468 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:40.839638 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:40.839830 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 12:21:40.839980 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4011,"thread_start_us":3871,"threads_started":1}
I20260809 12:21:40.841773 15087 tablet_service.cc:1467] Processing CreateTablet for tablet a51dbf9ce47e4170b8d4a0e5f46f55a8 (DEFAULT_TABLE table=test-table [id=6c074f31c4344cf1a4cf84c7b385fa24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:21:40.841776 15296 tablet_service.cc:1467] Processing CreateTablet for tablet a51dbf9ce47e4170b8d4a0e5f46f55a8 (DEFAULT_TABLE table=test-table [id=6c074f31c4344cf1a4cf84c7b385fa24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:21:40.843329 15087 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a51dbf9ce47e4170b8d4a0e5f46f55a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:40.843323 15296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a51dbf9ce47e4170b8d4a0e5f46f55a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:40.843706 14882 tablet_service.cc:1467] Processing CreateTablet for tablet a51dbf9ce47e4170b8d4a0e5f46f55a8 (DEFAULT_TABLE table=test-table [id=6c074f31c4344cf1a4cf84c7b385fa24]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:21:40.844800 14882 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a51dbf9ce47e4170b8d4a0e5f46f55a8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:40.855806 15433 tablet_bootstrap.cc:492] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 822633ef5f694d8580916a4f0c5ee400: Bootstrap starting.
I20260809 12:21:40.858093 15434 tablet_bootstrap.cc:492] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 280ec55393c142aa9b571aa095b0ed13: Bootstrap starting.
I20260809 12:21:40.858862 15435 tablet_bootstrap.cc:492] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590: Bootstrap starting.
I20260809 12:21:40.859576 15433 tablet_bootstrap.cc:654] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 822633ef5f694d8580916a4f0c5ee400: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:40.860921 15433 log.cc:824] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 822633ef5f694d8580916a4f0c5ee400: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:40.863157 15434 tablet_bootstrap.cc:654] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 280ec55393c142aa9b571aa095b0ed13: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:40.863708 15433 tablet_bootstrap.cc:492] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 822633ef5f694d8580916a4f0c5ee400: No bootstrap required, opened a new log
I20260809 12:21:40.863974 15433 ts_tablet_manager.cc:1397] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 822633ef5f694d8580916a4f0c5ee400: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260809 12:21:40.864413 15434 log.cc:824] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 280ec55393c142aa9b571aa095b0ed13: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:40.864466 15435 tablet_bootstrap.cc:654] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:40.865770 15435 log.cc:824] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:40.868278 15434 tablet_bootstrap.cc:492] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 280ec55393c142aa9b571aa095b0ed13: No bootstrap required, opened a new log
I20260809 12:21:40.868616 15434 ts_tablet_manager.cc:1397] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 280ec55393c142aa9b571aa095b0ed13: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260809 12:21:40.872349 15433 raft_consensus.cc:348] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 822633ef5f694d8580916a4f0c5ee400 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26a12e60cc8547759a9c0ecb00261590" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 38369 } } peers { permanent_uuid: "822633ef5f694d8580916a4f0c5ee400" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 35715 } } peers { permanent_uuid: "280ec55393c142aa9b571aa095b0ed13" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 39719 } }
I20260809 12:21:40.872709 15433 raft_consensus.cc:374] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 822633ef5f694d8580916a4f0c5ee400 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:40.872838 15433 raft_consensus.cc:729] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 822633ef5f694d8580916a4f0c5ee400 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 822633ef5f694d8580916a4f0c5ee400, State: Initialized, Role: FOLLOWER
I20260809 12:21:40.873422 15435 tablet_bootstrap.cc:492] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590: No bootstrap required, opened a new log
I20260809 12:21:40.873670 15435 ts_tablet_manager.cc:1397] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590: Time spent bootstrapping tablet: real 0.015s user 0.005s sys 0.005s
I20260809 12:21:40.873574 15433 consensus_queue.cc:260] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 822633ef5f694d8580916a4f0c5ee400 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26a12e60cc8547759a9c0ecb00261590" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 38369 } } peers { permanent_uuid: "822633ef5f694d8580916a4f0c5ee400" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 35715 } } peers { permanent_uuid: "280ec55393c142aa9b571aa095b0ed13" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 39719 } }
I20260809 12:21:40.879977 15433 ts_tablet_manager.cc:1428] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 822633ef5f694d8580916a4f0c5ee400: Time spent starting tablet: real 0.016s user 0.011s sys 0.005s
I20260809 12:21:40.879973 15434 raft_consensus.cc:348] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 280ec55393c142aa9b571aa095b0ed13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26a12e60cc8547759a9c0ecb00261590" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 38369 } } peers { permanent_uuid: "822633ef5f694d8580916a4f0c5ee400" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 35715 } } peers { permanent_uuid: "280ec55393c142aa9b571aa095b0ed13" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 39719 } }
I20260809 12:21:40.880429 15434 raft_consensus.cc:374] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 280ec55393c142aa9b571aa095b0ed13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:40.880551 15434 raft_consensus.cc:729] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 280ec55393c142aa9b571aa095b0ed13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 280ec55393c142aa9b571aa095b0ed13, State: Initialized, Role: FOLLOWER
I20260809 12:21:40.881215 15434 consensus_queue.cc:260] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 280ec55393c142aa9b571aa095b0ed13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26a12e60cc8547759a9c0ecb00261590" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 38369 } } peers { permanent_uuid: "822633ef5f694d8580916a4f0c5ee400" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 35715 } } peers { permanent_uuid: "280ec55393c142aa9b571aa095b0ed13" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 39719 } }
I20260809 12:21:40.882437 15187 heartbeater.cc:503] Master 127.19.183.254:42639 was elected leader, sending a full tablet report...
I20260809 12:21:40.883092 15434 ts_tablet_manager.cc:1428] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 280ec55393c142aa9b571aa095b0ed13: Time spent starting tablet: real 0.014s user 0.015s sys 0.001s
W20260809 12:21:40.884399 14987 tablet.cc:2369] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 822633ef5f694d8580916a4f0c5ee400: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:21:40.885087 15435 raft_consensus.cc:348] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26a12e60cc8547759a9c0ecb00261590" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 38369 } } peers { permanent_uuid: "822633ef5f694d8580916a4f0c5ee400" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 35715 } } peers { permanent_uuid: "280ec55393c142aa9b571aa095b0ed13" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 39719 } }
I20260809 12:21:40.885553 15435 raft_consensus.cc:374] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:40.885694 15435 raft_consensus.cc:729] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26a12e60cc8547759a9c0ecb00261590, State: Initialized, Role: FOLLOWER
I20260809 12:21:40.886317 15435 consensus_queue.cc:260] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26a12e60cc8547759a9c0ecb00261590" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 38369 } } peers { permanent_uuid: "822633ef5f694d8580916a4f0c5ee400" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 35715 } } peers { permanent_uuid: "280ec55393c142aa9b571aa095b0ed13" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 39719 } }
I20260809 12:21:40.887578 15404 heartbeater.cc:503] Master 127.19.183.254:42639 was elected leader, sending a full tablet report...
I20260809 12:21:40.887948 15435 ts_tablet_manager.cc:1428] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590: Time spent starting tablet: real 0.014s user 0.010s sys 0.004s
I20260809 12:21:40.899637 15444 raft_consensus.cc:482] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:21:40.899909 15444 raft_consensus.cc:504] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26a12e60cc8547759a9c0ecb00261590" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 38369 } } peers { permanent_uuid: "822633ef5f694d8580916a4f0c5ee400" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 35715 } } peers { permanent_uuid: "280ec55393c142aa9b571aa095b0ed13" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 39719 } }
I20260809 12:21:40.901574 15444 leader_election.cc:288] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 822633ef5f694d8580916a4f0c5ee400 (127.19.183.193:35715), 280ec55393c142aa9b571aa095b0ed13 (127.19.183.194:39719)
I20260809 12:21:40.908881 15431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:40.901948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:34003 (local address 127.19.183.193:35715)
0809 12:21:40.902154 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:40.902163 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:40.902479 (+ 316us) server_negotiation.cc:407] Connection header received
0809 12:21:40.902695 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:40.902732 (+ 37us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:40.902806 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:40.902883 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:40.903343 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.904384 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.906663 (+ 2279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.907863 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.908035 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:40.908111 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:40.908464 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 12:21:40.908576 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:21:40.908885 15410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:40.901984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:35715 (local address 127.19.183.195:34003)
0809 12:21:40.902346 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:40.902373 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:40.902519 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:40.902943 (+ 424us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:40.902948 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:40.902977 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:40.903205 (+ 228us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:40.903220 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.904528 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.904531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:40.906502 (+ 1971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:40.906522 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.908035 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.908044 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:40.908206 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:40.908230 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 12:21:40.908429 (+ 199us) client_negotiation.cc:240] Negotiation successful
0809 12:21:40.908566 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":220,"spinlock_wait_cycles":17792}
I20260809 12:21:40.909226 15430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:40.902022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:34531 (local address 127.19.183.194:39719)
0809 12:21:40.902288 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:40.902296 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:40.902977 (+ 681us) server_negotiation.cc:407] Connection header received
0809 12:21:40.903193 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:40.903197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:40.903252 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:40.903364 (+ 112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:40.903859 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.904785 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.907054 (+ 2269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.908229 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.908431 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:40.908492 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:40.908824 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 12:21:40.908967 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260809 12:21:40.909512 15445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:40.902038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:39719 (local address 127.19.183.195:34531)
0809 12:21:40.902842 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:40.902878 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:40.903054 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:40.903391 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:40.903394 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:40.903414 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:40.903696 (+ 282us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:40.903703 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.904944 (+ 1241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.904951 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:40.906909 (+ 1958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:40.906921 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.908460 (+ 1539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.908465 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:40.908610 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:40.908659 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 12:21:40.908762 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 12:21:40.908867 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":703,"spinlock_wait_cycles":21376,"thread_start_us":433,"threads_started":1}
I20260809 12:21:40.910090 14907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a51dbf9ce47e4170b8d4a0e5f46f55a8" candidate_uuid: "26a12e60cc8547759a9c0ecb00261590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "822633ef5f694d8580916a4f0c5ee400" is_pre_election: true
I20260809 12:21:40.910450 15121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a51dbf9ce47e4170b8d4a0e5f46f55a8" candidate_uuid: "26a12e60cc8547759a9c0ecb00261590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "280ec55393c142aa9b571aa095b0ed13" is_pre_election: true
I20260809 12:21:40.910588 14907 raft_consensus.cc:2454] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 822633ef5f694d8580916a4f0c5ee400 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26a12e60cc8547759a9c0ecb00261590 in term 0.
I20260809 12:21:40.910934 15121 raft_consensus.cc:2454] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 280ec55393c142aa9b571aa095b0ed13 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26a12e60cc8547759a9c0ecb00261590 in term 0.
I20260809 12:21:40.911347 15224 leader_election.cc:302] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26a12e60cc8547759a9c0ecb00261590, 822633ef5f694d8580916a4f0c5ee400; no voters:
I20260809 12:21:40.911734 15444 raft_consensus.cc:2781] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:21:40.911849 15444 raft_consensus.cc:482] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:21:40.911921 15444 raft_consensus.cc:3037] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:40.914039 15444 raft_consensus.cc:504] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26a12e60cc8547759a9c0ecb00261590" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 38369 } } peers { permanent_uuid: "822633ef5f694d8580916a4f0c5ee400" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 35715 } } peers { permanent_uuid: "280ec55393c142aa9b571aa095b0ed13" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 39719 } }
I20260809 12:21:40.914741 15444 leader_election.cc:288] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590 [CANDIDATE]: Term 1 election: Requested vote from peers 822633ef5f694d8580916a4f0c5ee400 (127.19.183.193:35715), 280ec55393c142aa9b571aa095b0ed13 (127.19.183.194:39719)
I20260809 12:21:40.915136 14907 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a51dbf9ce47e4170b8d4a0e5f46f55a8" candidate_uuid: "26a12e60cc8547759a9c0ecb00261590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "822633ef5f694d8580916a4f0c5ee400"
I20260809 12:21:40.915179 15121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a51dbf9ce47e4170b8d4a0e5f46f55a8" candidate_uuid: "26a12e60cc8547759a9c0ecb00261590" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "280ec55393c142aa9b571aa095b0ed13"
I20260809 12:21:40.915349 14907 raft_consensus.cc:3037] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 822633ef5f694d8580916a4f0c5ee400 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:40.915349 15121 raft_consensus.cc:3037] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 280ec55393c142aa9b571aa095b0ed13 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:40.918521 15121 raft_consensus.cc:2454] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 280ec55393c142aa9b571aa095b0ed13 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26a12e60cc8547759a9c0ecb00261590 in term 1.
I20260809 12:21:40.918521 14907 raft_consensus.cc:2454] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 822633ef5f694d8580916a4f0c5ee400 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26a12e60cc8547759a9c0ecb00261590 in term 1.
I20260809 12:21:40.919016 15224 leader_election.cc:302] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 26a12e60cc8547759a9c0ecb00261590, 280ec55393c142aa9b571aa095b0ed13, 822633ef5f694d8580916a4f0c5ee400; no voters:
I20260809 12:21:40.919340 15444 raft_consensus.cc:2781] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:21:40.919795 15444 raft_consensus.cc:686] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590 [term 1 LEADER]: Becoming Leader. State: Replica: 26a12e60cc8547759a9c0ecb00261590, State: Running, Role: LEADER
I20260809 12:21:40.920292 15444 consensus_queue.cc:237] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26a12e60cc8547759a9c0ecb00261590" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 38369 } } peers { permanent_uuid: "822633ef5f694d8580916a4f0c5ee400" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 35715 } } peers { permanent_uuid: "280ec55393c142aa9b571aa095b0ed13" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 39719 } }
I20260809 12:21:40.924978 14697 catalog_manager.cc:5495] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590 reported cstate change: term changed from 0 to 1, leader changed from <none> to 26a12e60cc8547759a9c0ecb00261590 (127.19.183.195). New cstate: current_term: 1 leader_uuid: "26a12e60cc8547759a9c0ecb00261590" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26a12e60cc8547759a9c0ecb00261590" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 38369 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "822633ef5f694d8580916a4f0c5ee400" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 35715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "280ec55393c142aa9b571aa095b0ed13" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 39719 } health_report { overall_health: UNKNOWN } } }
I20260809 12:21:40.947499 15427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:40.942114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33562 (local address 127.19.183.195:38369)
0809 12:21:40.942292 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:40.942296 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:40.942438 (+ 142us) server_negotiation.cc:407] Connection header received
0809 12:21:40.942592 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:40.942597 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:40.942681 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:40.942859 (+ 178us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:21:40.943322 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.944034 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.945407 (+ 1373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:40.945762 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:40.945846 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:40.946319 (+ 473us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:21:40.946935 (+ 616us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:21:40.947073 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:40.947240 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 12:21:40.947278 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260809 12:21:40.947707 15188 tablet.cc:2369] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 280ec55393c142aa9b571aa095b0ed13: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:21:40.955371 15121 raft_consensus.cc:1261] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 280ec55393c142aa9b571aa095b0ed13 [term 1 FOLLOWER]: Refusing update from remote peer 26a12e60cc8547759a9c0ecb00261590: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:21:40.955384 14907 raft_consensus.cc:1261] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 822633ef5f694d8580916a4f0c5ee400 [term 1 FOLLOWER]: Refusing update from remote peer 26a12e60cc8547759a9c0ecb00261590: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:21:40.956285 15447 consensus_queue.cc:1035] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590 [LEADER]: Connected to new peer: Peer: permanent_uuid: "280ec55393c142aa9b571aa095b0ed13" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 39719 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:21:40.956643 15444 consensus_queue.cc:1035] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590 [LEADER]: Connected to new peer: Peer: permanent_uuid: "822633ef5f694d8580916a4f0c5ee400" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 35715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:21:40.967747 15450 mvcc.cc:205] Tried to move back new op lower bound from 7316595101503717376 to 7316595101373181952. Current Snapshot: MvccSnapshot[applied={T|T < 7316595101503717376}]
I20260809 12:21:40.970188 15454 mvcc.cc:205] Tried to move back new op lower bound from 7316595101503717376 to 7316595101373181952. Current Snapshot: MvccSnapshot[applied={T|T < 7316595101503717376}]
I20260809 12:21:40.987838 14697 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:21:40.988142 14697 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51290:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:21:40.992053 14697 catalog_manager.cc:5782] T 00000000000000000000000000000000 P d7e4550d23cf40aba2f67525cc24f2de: Sending DeleteTablet for 3 replicas of tablet a51dbf9ce47e4170b8d4a0e5f46f55a8
I20260809 12:21:40.993115 15296 tablet_service.cc:1514] Processing DeleteTablet for tablet a51dbf9ce47e4170b8d4a0e5f46f55a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:21:40 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:33556
I20260809 12:21:40.993196 15087 tablet_service.cc:1514] Processing DeleteTablet for tablet a51dbf9ce47e4170b8d4a0e5f46f55a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:21:40 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:58086
I20260809 12:21:40.993222 14882 tablet_service.cc:1514] Processing DeleteTablet for tablet a51dbf9ce47e4170b8d4a0e5f46f55a8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:21:40 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:59712
I20260809 12:21:40.996021 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 14788
I20260809 12:21:41.006407 15472 tablet_replica.cc:330] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 822633ef5f694d8580916a4f0c5ee400: stopping tablet replica
I20260809 12:21:41.007220 15473 tablet_replica.cc:330] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 280ec55393c142aa9b571aa095b0ed13: stopping tablet replica
I20260809 12:21:41.007674 15472 raft_consensus.cc:2229] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 822633ef5f694d8580916a4f0c5ee400 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:21:41.008219 15472 raft_consensus.cc:2258] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 822633ef5f694d8580916a4f0c5ee400 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:21:41.008677 15473 raft_consensus.cc:2229] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 280ec55393c142aa9b571aa095b0ed13 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:21:41.009786 15473 raft_consensus.cc:2258] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 280ec55393c142aa9b571aa095b0ed13 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:21:41.010488 15472 ts_tablet_manager.cc:1905] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 822633ef5f694d8580916a4f0c5ee400: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:21:41.012205 15471 tablet_replica.cc:330] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590: stopping tablet replica
I20260809 12:21:41.012609 15471 raft_consensus.cc:2229] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:21:41.012974 15473 ts_tablet_manager.cc:1905] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 280ec55393c142aa9b571aa095b0ed13: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:21:41.014393 15471 raft_consensus.cc:2258] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:21:41.016379 15471 ts_tablet_manager.cc:1905] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:21:41.019636 15472 ts_tablet_manager.cc:1918] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 822633ef5f694d8580916a4f0c5ee400: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:21:41.019851 15472 log.cc:1196] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 822633ef5f694d8580916a4f0c5ee400: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/a51dbf9ce47e4170b8d4a0e5f46f55a8
I20260809 12:21:41.020447 15472 ts_tablet_manager.cc:1939] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 822633ef5f694d8580916a4f0c5ee400: Deleting consensus metadata
I20260809 12:21:41.020995 15473 ts_tablet_manager.cc:1918] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 280ec55393c142aa9b571aa095b0ed13: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:21:41.021234 15473 log.cc:1196] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 280ec55393c142aa9b571aa095b0ed13: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/a51dbf9ce47e4170b8d4a0e5f46f55a8
I20260809 12:21:41.021836 15473 ts_tablet_manager.cc:1939] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 280ec55393c142aa9b571aa095b0ed13: Deleting consensus metadata
Aug 09 12:21:41 dist-test-slave-39l8 krb5kdc[14632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.193@KRBTEST.COM
I20260809 12:21:41.022363 14677 catalog_manager.cc:4841] TS 822633ef5f694d8580916a4f0c5ee400 (127.19.183.193:35715): tablet a51dbf9ce47e4170b8d4a0e5f46f55a8 (table test-table [id=6c074f31c4344cf1a4cf84c7b385fa24]) successfully deleted
I20260809 12:21:41.023891 14678 catalog_manager.cc:4841] TS 280ec55393c142aa9b571aa095b0ed13 (127.19.183.194:39719): tablet a51dbf9ce47e4170b8d4a0e5f46f55a8 (table test-table [id=6c074f31c4344cf1a4cf84c7b385fa24]) successfully deleted
I20260809 12:21:41.026546 15471 ts_tablet_manager.cc:1918] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:21:41.026743 15471 log.cc:1196] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/a51dbf9ce47e4170b8d4a0e5f46f55a8
I20260809 12:21:41.027323 15471 ts_tablet_manager.cc:1939] T a51dbf9ce47e4170b8d4a0e5f46f55a8 P 26a12e60cc8547759a9c0ecb00261590: Deleting consensus metadata
I20260809 12:21:41.029049 14677 catalog_manager.cc:4841] TS 26a12e60cc8547759a9c0ecb00261590 (127.19.183.195:38369): tablet a51dbf9ce47e4170b8d4a0e5f46f55a8 (table test-table [id=6c074f31c4344cf1a4cf84c7b385fa24]) successfully deleted
I20260809 12:21:41.029524 15431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:40.996977 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59728 (local address 127.19.183.193:35715)
0809 12:21:40.997215 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:40.997225 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:41.007383 (+ 10158us) server_negotiation.cc:407] Connection header received
0809 12:21:41.007667 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:41.007674 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:41.007784 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:41.007893 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:41.009768 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:41.010860 (+ 1092us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:41.012767 (+ 1907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:41.013460 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:41.013569 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:41.023326 (+ 9757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:41.023339 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:41.023356 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:41.023403 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:41.026616 (+ 3213us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:41.027422 (+ 806us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:41.027433 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:41.027437 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:41.027590 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:41.027986 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:41.027991 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:41.027995 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:41.028378 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:41.028677 (+ 299us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:41.029036 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 12:21:41.029269 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 12:21:41.030854 14966 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:41.122354 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 14788
I20260809 12:21:41.149544 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 14994
Aug 09 12:21:41 dist-test-slave-39l8 krb5kdc[14632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.194@KRBTEST.COM
I20260809 12:21:41.166648 15430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:41.150454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58100 (local address 127.19.183.194:39719)
0809 12:21:41.150665 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:41.150671 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:41.150693 (+ 22us) server_negotiation.cc:407] Connection header received
0809 12:21:41.150876 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:41.150884 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:41.150988 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:41.151108 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:41.152394 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:41.153482 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:41.154852 (+ 1370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:41.155522 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:41.155674 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:41.159904 (+ 4230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:41.159924 (+ 20us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:41.159946 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:41.159989 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:41.163898 (+ 3909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:41.164557 (+ 659us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:41.164568 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:41.164571 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:41.164718 (+ 147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:41.165094 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:41.165099 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:41.165103 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:41.165575 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:41.165873 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:41.166040 (+ 167us) server_negotiation.cc:299] Negotiation successful
0809 12:21:41.166277 (+ 237us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260809 12:21:41.167439 15181 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:41.256477 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 14994
I20260809 12:21:41.281265 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 15196
Aug 09 12:21:41 dist-test-slave-39l8 krb5kdc[14632](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278098, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.195@KRBTEST.COM
I20260809 12:21:41.301151 15427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:41.282198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33572 (local address 127.19.183.195:38369)
0809 12:21:41.282387 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:41.282391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:41.282409 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:21:41.282666 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:41.282676 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:41.282797 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:41.282877 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:41.284822 (+ 1945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:41.285591 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:41.287608 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:41.288055 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:41.288154 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:41.293873 (+ 5719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:41.293885 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:41.293901 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:41.293936 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:41.297779 (+ 3843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:41.298422 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:41.298432 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:41.298436 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:41.298651 (+ 215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:41.299132 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:41.299137 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:41.299140 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:41.299627 (+ 487us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:41.299940 (+ 313us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:41.300295 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 12:21:41.300655 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260809 12:21:41.301954 15396 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:41.425851 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 15196
I20260809 12:21:41.456135 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 14653
I20260809 12:21:41.457252 14741 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:41.565898 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 14653
2026-08-09T12:21:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (4679 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T12:21:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:21:41Z Disabled control of system clock
I20260809 12:21:41.667697 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:44081
--webserver_interface=127.19.183.254
--webserver_port=0
--builtin_ntp_servers=127.19.183.212:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:44081
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 12:21:42.010123 15485 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:42.010459 15485 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:42.010664 15485 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:42.021981 15485 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:21:42.022089 15485 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:42.022127 15485 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:42.022185 15485 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:21:42.022229 15485 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:21:42.031482 15485 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:44081
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:44081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:42.032732 15485 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:42.034514 15485 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:42.047536 15492 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:42.047586 15493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:42.047645 15495 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:42.049003 15485 server_base.cc:1034] running on GCE node
I20260809 12:21:42.049798 15485 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:42.051304 15485 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:42.052556 15485 hybrid_clock.cc:648] HybridClock initialized: now 1786278102052528 us; error 40 us; skew 500 ppm
I20260809 12:21:42.052912 15485 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:21:42.055233 15485 webserver.cc:460] Webserver started at http://127.19.183.254:33491/ using document root <none> and password file <none>
I20260809 12:21:42.055958 15485 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:42.056097 15485 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:42.056432 15485 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:42.059430 15485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "4f37c53b8c6c4c54b0f2efb54265ba96"
format_stamp: "Formatted at 2026-08-09 12:21:42 on dist-test-slave-39l8"
server_key: "1e57dec46bcc1fcd021e20cb106f2ec0"
server_key_iv: "0dc01e879b92d7c9de7fd99183e94abe"
server_key_version: "encryptionkey@0"
I20260809 12:21:42.060302 15485 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "4f37c53b8c6c4c54b0f2efb54265ba96"
format_stamp: "Formatted at 2026-08-09 12:21:42 on dist-test-slave-39l8"
server_key: "1e57dec46bcc1fcd021e20cb106f2ec0"
server_key_iv: "0dc01e879b92d7c9de7fd99183e94abe"
server_key_version: "encryptionkey@0"
I20260809 12:21:42.065526 15485 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:21:42.069012 15503 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:42.070336 15485 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260809 12:21:42.070533 15485 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "4f37c53b8c6c4c54b0f2efb54265ba96"
format_stamp: "Formatted at 2026-08-09 12:21:42 on dist-test-slave-39l8"
server_key: "1e57dec46bcc1fcd021e20cb106f2ec0"
server_key_iv: "0dc01e879b92d7c9de7fd99183e94abe"
server_key_version: "encryptionkey@0"
I20260809 12:21:42.070776 15485 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:42.085633 15485 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:42.086504 15485 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:42.086895 15485 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:42.106217 15485 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:44081
I20260809 12:21:42.106237 15579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:44081 every 8 connection(s)
I20260809 12:21:42.107731 15485 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:21:42.111614 15580 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:42.116267 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 15485
I20260809 12:21:42.116492 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:21:42.116973 20191 external_mini_cluster.cc:1428] Setting key 347df4ee41e635e728340ae13a4504ea
I20260809 12:21:42.123005 15580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96: Bootstrap starting.
I20260809 12:21:42.126103 15580 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:42.127281 15580 log.cc:824] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:42.130939 15580 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96: No bootstrap required, opened a new log
I20260809 12:21:42.131373 15585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:42.119035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56816 (local address 127.19.183.254:44081)
0809 12:21:42.120153 (+ 1118us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:42.120164 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:42.120214 (+ 50us) server_negotiation.cc:407] Connection header received
0809 12:21:42.122007 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:42.122031 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:42.122483 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:42.122952 (+ 469us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:42.123547 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:42.124853 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:42.126285 (+ 1432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:42.127047 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:42.127266 (+ 219us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:42.128553 (+ 1287us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:42.128561 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:42.128574 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:42.129268 (+ 694us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:42.129374 (+ 106us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:42.129402 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:42.129526 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:42.129764 (+ 238us) server_negotiation.cc:299] Negotiation successful
0809 12:21:42.130095 (+ 331us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":687,"thread_start_us":414,"threads_started":1}
I20260809 12:21:42.139030 15580 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f37c53b8c6c4c54b0f2efb54265ba96" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44081 } }
I20260809 12:21:42.139372 15580 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:42.139446 15580 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f37c53b8c6c4c54b0f2efb54265ba96, State: Initialized, Role: FOLLOWER
I20260809 12:21:42.139982 15580 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f37c53b8c6c4c54b0f2efb54265ba96" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44081 } }
I20260809 12:21:42.140254 15580 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:21:42.140410 15580 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:21:42.140637 15580 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:42.142879 15580 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f37c53b8c6c4c54b0f2efb54265ba96" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44081 } }
I20260809 12:21:42.143402 15580 leader_election.cc:302] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f37c53b8c6c4c54b0f2efb54265ba96; no voters:
I20260809 12:21:42.144008 15580 leader_election.cc:288] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:21:42.144196 15589 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:21:42.145500 15589 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96 [term 1 LEADER]: Becoming Leader. State: Replica: 4f37c53b8c6c4c54b0f2efb54265ba96, State: Running, Role: LEADER
I20260809 12:21:42.146183 15580 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:21:42.145978 15589 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f37c53b8c6c4c54b0f2efb54265ba96" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44081 } }
I20260809 12:21:42.149127 15592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f37c53b8c6c4c54b0f2efb54265ba96. Latest consensus state: current_term: 1 leader_uuid: "4f37c53b8c6c4c54b0f2efb54265ba96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f37c53b8c6c4c54b0f2efb54265ba96" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44081 } } }
I20260809 12:21:42.149394 15590 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f37c53b8c6c4c54b0f2efb54265ba96" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f37c53b8c6c4c54b0f2efb54265ba96" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 44081 } } }
I20260809 12:21:42.149626 15592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96 [sys.catalog]: This master's current role is: LEADER
I20260809 12:21:42.149698 15590 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96 [sys.catalog]: This master's current role is: LEADER
I20260809 12:21:42.152693 15603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:21:42.158010 15603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:21:42.166565 15603 catalog_manager.cc:1347] Generated new cluster ID: a3c604e6eae54a3eb9697e4cc4915749
I20260809 12:21:42.166690 15603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:21:42.188460 15603 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:21:42.189779 15603 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:21:42.201792 15603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96: Generated new TSK 0
I20260809 12:21:42.202685 15603 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:21:42.218447 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:44081
--builtin_ntp_servers=127.19.183.212:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 12:21:42.579975 15614 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:42.580251 15614 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:42.580338 15614 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:42.587291 15614 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:42.587378 15614 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:42.587477 15614 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:21:42.596312 15614 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:44081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:42.597570 15614 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:42.599330 15614 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:42.611663 15627 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:42.611848 15628 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:42.611956 15630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:42.613059 15614 server_base.cc:1034] running on GCE node
I20260809 12:21:42.613636 15614 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:42.614552 15614 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:42.615782 15614 hybrid_clock.cc:648] HybridClock initialized: now 1786278102615709 us; error 65 us; skew 500 ppm
I20260809 12:21:42.616099 15614 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:21:42.618289 15614 webserver.cc:460] Webserver started at http://127.19.183.193:44365/ using document root <none> and password file <none>
I20260809 12:21:42.619046 15614 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:42.619181 15614 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:42.619495 15614 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:42.622251 15614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "b1862e4707474c028bacb22100da7091"
format_stamp: "Formatted at 2026-08-09 12:21:42 on dist-test-slave-39l8"
server_key: "a529cda2cf68751524372a8fa2cce37f"
server_key_iv: "2ac9c5392b9f003b2e8e199152238860"
server_key_version: "encryptionkey@0"
I20260809 12:21:42.623155 15614 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "b1862e4707474c028bacb22100da7091"
format_stamp: "Formatted at 2026-08-09 12:21:42 on dist-test-slave-39l8"
server_key: "a529cda2cf68751524372a8fa2cce37f"
server_key_iv: "2ac9c5392b9f003b2e8e199152238860"
server_key_version: "encryptionkey@0"
I20260809 12:21:42.628012 15614 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260809 12:21:42.631403 15638 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:42.632670 15614 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:21:42.632844 15614 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "b1862e4707474c028bacb22100da7091"
format_stamp: "Formatted at 2026-08-09 12:21:42 on dist-test-slave-39l8"
server_key: "a529cda2cf68751524372a8fa2cce37f"
server_key_iv: "2ac9c5392b9f003b2e8e199152238860"
server_key_version: "encryptionkey@0"
I20260809 12:21:42.633052 15614 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:42.646953 15614 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:42.647688 15614 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:42.647993 15614 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:42.648960 15614 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:21:42.650535 15614 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:21:42.650647 15614 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:42.650789 15614 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:21:42.650885 15614 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:42.684473 15614 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:40595
I20260809 12:21:42.684530 15803 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:40595 every 8 connection(s)
I20260809 12:21:42.685827 15614 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:21:42.688000 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 15614
I20260809 12:21:42.688213 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:21:42.688822 20191 external_mini_cluster.cc:1428] Setting key 8f03e788e5425f3f0e1d00a588e6c955
I20260809 12:21:42.691908 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:44081
--builtin_ntp_servers=127.19.183.212:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 12:21:42.700183 15806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:42.688328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:42313 (local address 127.19.183.254:44081)
0809 12:21:42.688828 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:42.688835 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:42.690130 (+ 1295us) server_negotiation.cc:407] Connection header received
0809 12:21:42.692647 (+ 2517us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:42.692651 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:42.692722 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:42.692834 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:42.694307 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:42.695084 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:42.697424 (+ 2340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:42.697826 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:42.697929 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:42.699451 (+ 1522us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:42.699456 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:42.699459 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:42.699476 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:42.699517 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:42.699528 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:42.699592 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:42.699863 (+ 271us) server_negotiation.cc:299] Negotiation successful
0809 12:21:42.699933 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":375,"thread_start_us":242,"threads_started":1}
I20260809 12:21:42.701637 15809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:42.688965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:44081 (local address 127.19.183.193:42313)
0809 12:21:42.689982 (+ 1017us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:42.690023 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:42.692362 (+ 2339us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:42.693126 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:42.693141 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:42.693272 (+ 131us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:21:42.694126 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:42.694155 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:42.695247 (+ 1092us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:42.695255 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:42.697283 (+ 2028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:42.697301 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:42.698058 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:42.698067 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:42.698372 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:42.699113 (+ 741us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:21:42.699118 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:21:42.699168 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:21:42.699263 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:21:42.699622 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:21:42.699635 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:21:42.699645 (+ 10us) client_negotiation.cc:769] Sending connection context
0809 12:21:42.699849 (+ 204us) client_negotiation.cc:240] Negotiation successful
0809 12:21:42.700244 (+ 395us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"thread_start_us":351,"threads_started":1}
I20260809 12:21:42.703114 15804 heartbeater.cc:347] Connected to a master server at 127.19.183.254:44081
I20260809 12:21:42.703400 15804 heartbeater.cc:464] Registering TS with master...
I20260809 12:21:42.704172 15804 heartbeater.cc:511] Master 127.19.183.254:44081 requested a full tablet report, sending...
I20260809 12:21:42.706171 15528 ts_manager.cc:194] Registered new tserver with Master: b1862e4707474c028bacb22100da7091 (127.19.183.193:40595)
I20260809 12:21:42.708213 15528 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.183.193:42313
W20260809 12:21:43.040469 15810 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:43.040746 15810 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:43.040832 15810 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:43.047935 15810 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:43.048035 15810 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:43.048142 15810 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:21:43.057317 15810 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:44081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:43.058566 15810 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:43.060386 15810 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:43.080072 15815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:43.080016 15816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:43.080098 15820 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:43.081355 15810 server_base.cc:1034] running on GCE node
I20260809 12:21:43.081974 15810 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:43.082988 15810 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:43.084194 15810 hybrid_clock.cc:648] HybridClock initialized: now 1786278103084142 us; error 49 us; skew 500 ppm
I20260809 12:21:43.084522 15810 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:21:43.086766 15810 webserver.cc:460] Webserver started at http://127.19.183.194:33211/ using document root <none> and password file <none>
I20260809 12:21:43.087508 15810 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:43.087670 15810 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:43.087992 15810 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:43.091033 15810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "49c5be84584b4209b0878618dbfde238"
format_stamp: "Formatted at 2026-08-09 12:21:43 on dist-test-slave-39l8"
server_key: "221cc6a683fbc8dd99ca9bffabe9b492"
server_key_iv: "1562c5846cd443e00b2dab0407172426"
server_key_version: "encryptionkey@0"
I20260809 12:21:43.091827 15810 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "49c5be84584b4209b0878618dbfde238"
format_stamp: "Formatted at 2026-08-09 12:21:43 on dist-test-slave-39l8"
server_key: "221cc6a683fbc8dd99ca9bffabe9b492"
server_key_iv: "1562c5846cd443e00b2dab0407172426"
server_key_version: "encryptionkey@0"
I20260809 12:21:43.096963 15810 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:21:43.100328 15829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:43.101624 15810 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:21:43.101828 15810 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "49c5be84584b4209b0878618dbfde238"
format_stamp: "Formatted at 2026-08-09 12:21:43 on dist-test-slave-39l8"
server_key: "221cc6a683fbc8dd99ca9bffabe9b492"
server_key_iv: "1562c5846cd443e00b2dab0407172426"
server_key_version: "encryptionkey@0"
I20260809 12:21:43.102046 15810 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:43.125979 15810 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:43.126924 15810 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:43.127231 15810 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:43.128352 15810 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:21:43.129745 15810 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:21:43.129839 15810 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:43.129916 15810 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:21:43.129973 15810 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:43.162098 15810 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:41775
I20260809 12:21:43.162149 15988 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:41775 every 8 connection(s)
I20260809 12:21:43.163463 15810 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:21:43.168579 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 15810
I20260809 12:21:43.168759 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:21:43.169210 20191 external_mini_cluster.cc:1428] Setting key 0836ec8ca9d1e2f7b3e0b1d581c39eb8
I20260809 12:21:43.172083 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:44081
--builtin_ntp_servers=127.19.183.212:43891
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 12:21:43.176644 15806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:43.166182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:34655 (local address 127.19.183.254:44081)
0809 12:21:43.166369 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:43.166374 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:43.167792 (+ 1418us) server_negotiation.cc:407] Connection header received
0809 12:21:43.169752 (+ 1960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:43.169758 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:43.169852 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:43.169969 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:43.171304 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.172039 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.173650 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.174102 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.174245 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:43.175782 (+ 1537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:43.175805 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:43.175816 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:43.175834 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:43.175866 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:43.175878 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:43.175976 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:43.176229 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 12:21:43.176330 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 12:21:43.177670 15991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:43.166637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:44081 (local address 127.19.183.194:34655)
0809 12:21:43.167625 (+ 988us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:43.167661 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:43.169470 (+ 1809us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:43.170234 (+ 764us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:43.170245 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:43.170357 (+ 112us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:21:43.171128 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:43.171148 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.172225 (+ 1077us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.172231 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:43.173499 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:43.173512 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.174369 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.174379 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:43.174605 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:43.175393 (+ 788us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:21:43.175396 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:21:43.175434 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:21:43.175583 (+ 149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:21:43.175981 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:21:43.175996 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:21:43.176005 (+ 9us) client_negotiation.cc:769] Sending connection context
0809 12:21:43.176207 (+ 202us) client_negotiation.cc:240] Negotiation successful
0809 12:21:43.176611 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":567,"thread_start_us":301,"threads_started":1}
I20260809 12:21:43.179123 15989 heartbeater.cc:347] Connected to a master server at 127.19.183.254:44081
I20260809 12:21:43.179435 15989 heartbeater.cc:464] Registering TS with master...
I20260809 12:21:43.180190 15989 heartbeater.cc:511] Master 127.19.183.254:44081 requested a full tablet report, sending...
I20260809 12:21:43.181767 15528 ts_manager.cc:194] Registered new tserver with Master: 49c5be84584b4209b0878618dbfde238 (127.19.183.194:41775)
I20260809 12:21:43.182981 15528 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.183.194:34655
W20260809 12:21:43.518538 15992 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:43.518831 15992 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:43.518916 15992 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:43.525808 15992 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:43.525910 15992 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:43.526007 15992 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:21:43.534505 15992 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:43891
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:44081
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:43.535766 15992 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:43.537626 15992 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:43.550932 15998 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:43.551033 16000 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:43.551182 15997 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:43.551570 15992 server_base.cc:1034] running on GCE node
I20260809 12:21:43.552230 15992 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:43.553205 15992 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:43.554437 15992 hybrid_clock.cc:648] HybridClock initialized: now 1786278103554389 us; error 44 us; skew 500 ppm
I20260809 12:21:43.554821 15992 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:21:43.557142 15992 webserver.cc:460] Webserver started at http://127.19.183.195:36881/ using document root <none> and password file <none>
I20260809 12:21:43.557832 15992 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:43.557963 15992 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:43.558290 15992 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:43.561239 15992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "c40b851c196f4005a4cf03ee41a8dd94"
format_stamp: "Formatted at 2026-08-09 12:21:43 on dist-test-slave-39l8"
server_key: "128976ab0fed954bc91fc1c71a5660f1"
server_key_iv: "e85621d5824ff496911e2ed795bbcfd2"
server_key_version: "encryptionkey@0"
I20260809 12:21:43.562031 15992 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "c40b851c196f4005a4cf03ee41a8dd94"
format_stamp: "Formatted at 2026-08-09 12:21:43 on dist-test-slave-39l8"
server_key: "128976ab0fed954bc91fc1c71a5660f1"
server_key_iv: "e85621d5824ff496911e2ed795bbcfd2"
server_key_version: "encryptionkey@0"
I20260809 12:21:43.567085 15992 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 12:21:43.570647 16011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:43.571897 15992 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:21:43.572094 15992 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "c40b851c196f4005a4cf03ee41a8dd94"
format_stamp: "Formatted at 2026-08-09 12:21:43 on dist-test-slave-39l8"
server_key: "128976ab0fed954bc91fc1c71a5660f1"
server_key_iv: "e85621d5824ff496911e2ed795bbcfd2"
server_key_version: "encryptionkey@0"
I20260809 12:21:43.572329 15992 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:43.587106 15992 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:43.587841 15992 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:43.588155 15992 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:43.589039 15992 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:21:43.590381 15992 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:21:43.590504 15992 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:43.590639 15992 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:21:43.590778 15992 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:43.624275 15992 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:39985
I20260809 12:21:43.624317 16186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:39985 every 8 connection(s)
I20260809 12:21:43.625582 15992 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:21:43.627199 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 15992
I20260809 12:21:43.627399 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:21:43.627841 20191 external_mini_cluster.cc:1428] Setting key 38a35c8125c7bf61e335ebed307c4adb
I20260809 12:21:43.645330 15806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:43.628243 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:58311 (local address 127.19.183.254:44081)
0809 12:21:43.628723 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:43.628729 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:43.630123 (+ 1394us) server_negotiation.cc:407] Connection header received
0809 12:21:43.635501 (+ 5378us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:43.635506 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:43.635606 (+ 100us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:43.635713 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:43.637681 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.639108 (+ 1427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.642282 (+ 3174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.642924 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.643118 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:43.644481 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:43.644487 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:43.644499 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:43.644517 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:43.644551 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:43.644562 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:43.644662 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:43.644897 (+ 235us) server_negotiation.cc:299] Negotiation successful
0809 12:21:43.644995 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331}
I20260809 12:21:43.646350 16190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:43.628843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:44081 (local address 127.19.183.195:58311)
0809 12:21:43.629957 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:43.630007 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:43.635251 (+ 5244us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:43.636167 (+ 916us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:43.636181 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:43.636338 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:21:43.637424 (+ 1086us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:43.637454 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.640268 (+ 2814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.640276 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:43.642137 (+ 1861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:43.642152 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.643208 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.643216 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:43.643496 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:43.644164 (+ 668us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:21:43.644167 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:21:43.644205 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:21:43.644277 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:21:43.644662 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:21:43.644676 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:21:43.644687 (+ 11us) client_negotiation.cc:769] Sending connection context
0809 12:21:43.644881 (+ 194us) client_negotiation.cc:240] Negotiation successful
0809 12:21:43.645258 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":584,"thread_start_us":362,"threads_started":1}
I20260809 12:21:43.647708 16187 heartbeater.cc:347] Connected to a master server at 127.19.183.254:44081
I20260809 12:21:43.648015 16187 heartbeater.cc:464] Registering TS with master...
I20260809 12:21:43.648744 16187 heartbeater.cc:511] Master 127.19.183.254:44081 requested a full tablet report, sending...
I20260809 12:21:43.650207 15528 ts_manager.cc:194] Registered new tserver with Master: c40b851c196f4005a4cf03ee41a8dd94 (127.19.183.195:39985)
I20260809 12:21:43.651211 15528 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.183.195:58311
I20260809 12:21:43.655599 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:21:43.665277 15806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:43.659743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56832 (local address 127.19.183.254:44081)
0809 12:21:43.659945 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:43.659954 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:43.660391 (+ 437us) server_negotiation.cc:407] Connection header received
0809 12:21:43.660509 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:43.660513 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:43.660582 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:43.660664 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:43.661074 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.661730 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.663012 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.663421 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.663511 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:43.664538 (+ 1027us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:43.664543 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:43.664546 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:43.664563 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:43.664593 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:43.664604 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:43.664715 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:43.664874 (+ 159us) server_negotiation.cc:299] Negotiation successful
0809 12:21:43.664951 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 12:21:43.668980 15528 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:56832:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:21:43.675896 15528 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:21:43.711938 15804 heartbeater.cc:503] Master 127.19.183.254:44081 was elected leader, sending a full tablet report...
I20260809 12:21:43.714310 16215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:43.688808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45796 (local address 127.19.183.195:39985)
0809 12:21:43.702009 (+ 13201us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:43.702021 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:43.704155 (+ 2134us) server_negotiation.cc:407] Connection header received
0809 12:21:43.704291 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:43.704303 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:43.704633 (+ 330us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:43.704848 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:43.706356 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.708261 (+ 1905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.710890 (+ 2629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.712558 (+ 1668us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.712743 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:43.712964 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:43.713695 (+ 731us) server_negotiation.cc:299] Negotiation successful
0809 12:21:43.713898 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12936,"thread_start_us":13204,"threads_started":1}
I20260809 12:21:43.714326 16209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:43.688424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:39985 (local address 127.0.0.1:45796)
0809 12:21:43.703807 (+ 15383us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:43.703828 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:43.703997 (+ 169us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:43.705300 (+ 1303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:43.705311 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:43.705352 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:43.705943 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:43.705960 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.708436 (+ 2476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.708441 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:43.710627 (+ 2186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:43.710642 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.713109 (+ 2467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.713115 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:43.713350 (+ 235us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:43.713404 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 12:21:43.713559 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 12:21:43.713896 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15225,"mutex_wait_us":69,"thread_start_us":15109,"threads_started":1}
I20260809 12:21:43.716934 16218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:43.690531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:41775 (local address 127.0.0.1:44848)
0809 12:21:43.707802 (+ 17271us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:43.707818 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:43.707923 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:43.709248 (+ 1325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:43.709252 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:43.709278 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:43.709792 (+ 514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:43.709805 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.711665 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.711670 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:43.713868 (+ 2198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:43.713882 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.716160 (+ 2278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.716168 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:43.716337 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:43.716373 (+ 36us) client_negotiation.cc:769] Sending connection context
0809 12:21:43.716467 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 12:21:43.716584 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":17163,"mutex_wait_us":1,"thread_start_us":17458,"threads_started":1}
I20260809 12:21:43.717079 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:43.690846 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44848 (local address 127.19.183.194:41775)
0809 12:21:43.708638 (+ 17792us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:43.708647 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:43.708677 (+ 30us) server_negotiation.cc:407] Connection header received
0809 12:21:43.708781 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:43.708788 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:43.709036 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:43.709243 (+ 207us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:43.709978 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.711502 (+ 1524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.714115 (+ 2613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.715643 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.715807 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:43.716035 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:43.716544 (+ 509us) server_negotiation.cc:299] Negotiation successful
0809 12:21:43.716740 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17611,"thread_start_us":17494,"threads_started":1}
I20260809 12:21:43.718259 16206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:43.688294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:40595 (local address 127.0.0.1:58628)
0809 12:21:43.702924 (+ 14630us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:43.702986 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:43.703207 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:43.709815 (+ 6608us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:43.709822 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:43.709851 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:43.710196 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:43.710204 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.712575 (+ 2371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.712579 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:43.715286 (+ 2707us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:43.715300 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.717499 (+ 2199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.717506 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:43.717730 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:43.717763 (+ 33us) client_negotiation.cc:769] Sending connection context
0809 12:21:43.717870 (+ 107us) client_negotiation.cc:240] Negotiation successful
0809 12:21:43.717981 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":14280,"thread_start_us":14101,"threads_started":1}
I20260809 12:21:43.718659 16214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:43.688920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58628 (local address 127.19.183.193:40595)
0809 12:21:43.709040 (+ 20120us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:43.709054 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:43.709076 (+ 22us) server_negotiation.cc:407] Connection header received
0809 12:21:43.709159 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:43.709163 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:43.709401 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:43.709610 (+ 209us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:43.710463 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.712389 (+ 1926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.715480 (+ 3091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.716946 (+ 1466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.717146 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:43.717362 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:43.717934 (+ 572us) server_negotiation.cc:299] Negotiation successful
0809 12:21:43.718094 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19969,"thread_start_us":19852,"threads_started":1}
I20260809 12:21:43.718827 16088 tablet_service.cc:1467] Processing CreateTablet for tablet 67cbbdd1aded432d85460413c7d1c868 (DEFAULT_TABLE table=test-table [id=22f417230ad54908b4d942d15fb69f15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:21:43.720651 16088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67cbbdd1aded432d85460413c7d1c868. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:43.721558 15899 tablet_service.cc:1467] Processing CreateTablet for tablet 67cbbdd1aded432d85460413c7d1c868 (DEFAULT_TABLE table=test-table [id=22f417230ad54908b4d942d15fb69f15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:21:43.722059 15705 tablet_service.cc:1467] Processing CreateTablet for tablet 67cbbdd1aded432d85460413c7d1c868 (DEFAULT_TABLE table=test-table [id=22f417230ad54908b4d942d15fb69f15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:21:43.722925 15899 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67cbbdd1aded432d85460413c7d1c868. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:43.723426 15705 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 67cbbdd1aded432d85460413c7d1c868. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:43.731896 16221 tablet_bootstrap.cc:492] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94: Bootstrap starting.
I20260809 12:21:43.735143 16221 tablet_bootstrap.cc:654] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:43.736184 16221 log.cc:824] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:43.738526 16224 tablet_bootstrap.cc:492] T 67cbbdd1aded432d85460413c7d1c868 P b1862e4707474c028bacb22100da7091: Bootstrap starting.
I20260809 12:21:43.739832 16221 tablet_bootstrap.cc:492] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94: No bootstrap required, opened a new log
I20260809 12:21:43.740141 16221 ts_tablet_manager.cc:1397] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.006s
I20260809 12:21:43.740628 16225 tablet_bootstrap.cc:492] T 67cbbdd1aded432d85460413c7d1c868 P 49c5be84584b4209b0878618dbfde238: Bootstrap starting.
I20260809 12:21:43.742108 16224 tablet_bootstrap.cc:654] T 67cbbdd1aded432d85460413c7d1c868 P b1862e4707474c028bacb22100da7091: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:43.743695 16225 tablet_bootstrap.cc:654] T 67cbbdd1aded432d85460413c7d1c868 P 49c5be84584b4209b0878618dbfde238: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:43.743752 16224 log.cc:824] T 67cbbdd1aded432d85460413c7d1c868 P b1862e4707474c028bacb22100da7091: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:43.744886 16225 log.cc:824] T 67cbbdd1aded432d85460413c7d1c868 P 49c5be84584b4209b0878618dbfde238: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:43.746894 16224 tablet_bootstrap.cc:492] T 67cbbdd1aded432d85460413c7d1c868 P b1862e4707474c028bacb22100da7091: No bootstrap required, opened a new log
I20260809 12:21:43.747145 16224 ts_tablet_manager.cc:1397] T 67cbbdd1aded432d85460413c7d1c868 P b1862e4707474c028bacb22100da7091: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.003s
I20260809 12:21:43.747637 16225 tablet_bootstrap.cc:492] T 67cbbdd1aded432d85460413c7d1c868 P 49c5be84584b4209b0878618dbfde238: No bootstrap required, opened a new log
I20260809 12:21:43.747920 16225 ts_tablet_manager.cc:1397] T 67cbbdd1aded432d85460413c7d1c868 P 49c5be84584b4209b0878618dbfde238: Time spent bootstrapping tablet: real 0.008s user 0.006s sys 0.000s
I20260809 12:21:43.751634 16221 raft_consensus.cc:348] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1862e4707474c028bacb22100da7091" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40595 } } peers { permanent_uuid: "c40b851c196f4005a4cf03ee41a8dd94" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 39985 } } peers { permanent_uuid: "49c5be84584b4209b0878618dbfde238" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41775 } }
I20260809 12:21:43.752131 16221 raft_consensus.cc:374] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:43.752240 16221 raft_consensus.cc:729] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c40b851c196f4005a4cf03ee41a8dd94, State: Initialized, Role: FOLLOWER
I20260809 12:21:43.752951 16221 consensus_queue.cc:260] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1862e4707474c028bacb22100da7091" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40595 } } peers { permanent_uuid: "c40b851c196f4005a4cf03ee41a8dd94" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 39985 } } peers { permanent_uuid: "49c5be84584b4209b0878618dbfde238" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41775 } }
I20260809 12:21:43.754351 16187 heartbeater.cc:503] Master 127.19.183.254:44081 was elected leader, sending a full tablet report...
I20260809 12:21:43.754853 16221 ts_tablet_manager.cc:1428] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94: Time spent starting tablet: real 0.015s user 0.007s sys 0.006s
I20260809 12:21:43.756065 16225 raft_consensus.cc:348] T 67cbbdd1aded432d85460413c7d1c868 P 49c5be84584b4209b0878618dbfde238 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1862e4707474c028bacb22100da7091" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40595 } } peers { permanent_uuid: "c40b851c196f4005a4cf03ee41a8dd94" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 39985 } } peers { permanent_uuid: "49c5be84584b4209b0878618dbfde238" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41775 } }
I20260809 12:21:43.756438 16225 raft_consensus.cc:374] T 67cbbdd1aded432d85460413c7d1c868 P 49c5be84584b4209b0878618dbfde238 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:43.756556 16225 raft_consensus.cc:729] T 67cbbdd1aded432d85460413c7d1c868 P 49c5be84584b4209b0878618dbfde238 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49c5be84584b4209b0878618dbfde238, State: Initialized, Role: FOLLOWER
I20260809 12:21:43.757153 16225 consensus_queue.cc:260] T 67cbbdd1aded432d85460413c7d1c868 P 49c5be84584b4209b0878618dbfde238 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1862e4707474c028bacb22100da7091" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40595 } } peers { permanent_uuid: "c40b851c196f4005a4cf03ee41a8dd94" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 39985 } } peers { permanent_uuid: "49c5be84584b4209b0878618dbfde238" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41775 } }
I20260809 12:21:43.758314 15989 heartbeater.cc:503] Master 127.19.183.254:44081 was elected leader, sending a full tablet report...
I20260809 12:21:43.758988 16225 ts_tablet_manager.cc:1428] T 67cbbdd1aded432d85460413c7d1c868 P 49c5be84584b4209b0878618dbfde238: Time spent starting tablet: real 0.011s user 0.012s sys 0.000s
I20260809 12:21:43.759150 16224 raft_consensus.cc:348] T 67cbbdd1aded432d85460413c7d1c868 P b1862e4707474c028bacb22100da7091 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1862e4707474c028bacb22100da7091" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40595 } } peers { permanent_uuid: "c40b851c196f4005a4cf03ee41a8dd94" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 39985 } } peers { permanent_uuid: "49c5be84584b4209b0878618dbfde238" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41775 } }
I20260809 12:21:43.759570 16224 raft_consensus.cc:374] T 67cbbdd1aded432d85460413c7d1c868 P b1862e4707474c028bacb22100da7091 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:43.759703 16224 raft_consensus.cc:729] T 67cbbdd1aded432d85460413c7d1c868 P b1862e4707474c028bacb22100da7091 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1862e4707474c028bacb22100da7091, State: Initialized, Role: FOLLOWER
I20260809 12:21:43.760751 16224 consensus_queue.cc:260] T 67cbbdd1aded432d85460413c7d1c868 P b1862e4707474c028bacb22100da7091 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1862e4707474c028bacb22100da7091" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40595 } } peers { permanent_uuid: "c40b851c196f4005a4cf03ee41a8dd94" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 39985 } } peers { permanent_uuid: "49c5be84584b4209b0878618dbfde238" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41775 } }
I20260809 12:21:43.762334 16224 ts_tablet_manager.cc:1428] T 67cbbdd1aded432d85460413c7d1c868 P b1862e4707474c028bacb22100da7091: Time spent starting tablet: real 0.015s user 0.012s sys 0.001s
I20260809 12:21:43.818496 16231 raft_consensus.cc:482] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:21:43.818825 16231 raft_consensus.cc:504] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1862e4707474c028bacb22100da7091" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40595 } } peers { permanent_uuid: "c40b851c196f4005a4cf03ee41a8dd94" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 39985 } } peers { permanent_uuid: "49c5be84584b4209b0878618dbfde238" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41775 } }
I20260809 12:21:43.820619 16231 leader_election.cc:288] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b1862e4707474c028bacb22100da7091 (127.19.183.193:40595), 49c5be84584b4209b0878618dbfde238 (127.19.183.194:41775)
I20260809 12:21:43.827157 16214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:43.821063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:37505 (local address 127.19.183.193:40595)
0809 12:21:43.821267 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:43.821274 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:43.821570 (+ 296us) server_negotiation.cc:407] Connection header received
0809 12:21:43.821655 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:43.821659 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:43.821771 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:43.821875 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:43.822429 (+ 554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.823352 (+ 923us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.825299 (+ 1947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.826184 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.826307 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:43.826382 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:43.826780 (+ 398us) server_negotiation.cc:299] Negotiation successful
0809 12:21:43.826910 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:21:43.827183 16190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:43.820926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:40595 (local address 127.19.183.195:37505)
0809 12:21:43.821255 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:43.821275 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:43.821423 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:43.821988 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:43.821994 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:43.822025 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:43.822225 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:43.822237 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.823495 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.823501 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:43.825120 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:43.825132 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.826341 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.826349 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:43.826542 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:43.826601 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 12:21:43.826673 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 12:21:43.826847 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":91}
I20260809 12:21:43.827188 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:43.821327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:44055 (local address 127.19.183.194:41775)
0809 12:21:43.821548 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:43.821553 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:43.821884 (+ 331us) server_negotiation.cc:407] Connection header received
0809 12:21:43.822035 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:43.822040 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:43.822133 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:43.822236 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:43.822932 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.823686 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.825511 (+ 1825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.826295 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.826440 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:43.826512 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:43.826823 (+ 311us) server_negotiation.cc:299] Negotiation successful
0809 12:21:43.826949 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 12:21:43.827279 16234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:43.821044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:41775 (local address 127.19.183.195:44055)
0809 12:21:43.821779 (+ 735us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:43.821796 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:43.821910 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:43.822284 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:43.822288 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:43.822307 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:43.822751 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:43.822767 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.823842 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.823846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:43.825301 (+ 1455us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:43.825315 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.826471 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.826480 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:43.826632 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:43.826676 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 12:21:43.826792 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 12:21:43.826910 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":621,"mutex_wait_us":40,"thread_start_us":393,"threads_started":1}
I20260809 12:21:43.828279 15923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67cbbdd1aded432d85460413c7d1c868" candidate_uuid: "c40b851c196f4005a4cf03ee41a8dd94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c5be84584b4209b0878618dbfde238" is_pre_election: true
I20260809 12:21:43.828707 15736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67cbbdd1aded432d85460413c7d1c868" candidate_uuid: "c40b851c196f4005a4cf03ee41a8dd94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1862e4707474c028bacb22100da7091" is_pre_election: true
I20260809 12:21:43.828917 15923 raft_consensus.cc:2454] T 67cbbdd1aded432d85460413c7d1c868 P 49c5be84584b4209b0878618dbfde238 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40b851c196f4005a4cf03ee41a8dd94 in term 0.
I20260809 12:21:43.829289 15736 raft_consensus.cc:2454] T 67cbbdd1aded432d85460413c7d1c868 P b1862e4707474c028bacb22100da7091 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c40b851c196f4005a4cf03ee41a8dd94 in term 0.
I20260809 12:21:43.829720 16012 leader_election.cc:302] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 49c5be84584b4209b0878618dbfde238, c40b851c196f4005a4cf03ee41a8dd94; no voters:
I20260809 12:21:43.830161 16231 raft_consensus.cc:2781] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:21:43.830339 16231 raft_consensus.cc:482] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:21:43.830453 16231 raft_consensus.cc:3037] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:43.832784 16231 raft_consensus.cc:504] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1862e4707474c028bacb22100da7091" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40595 } } peers { permanent_uuid: "c40b851c196f4005a4cf03ee41a8dd94" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 39985 } } peers { permanent_uuid: "49c5be84584b4209b0878618dbfde238" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41775 } }
I20260809 12:21:43.833379 16231 leader_election.cc:288] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94 [CANDIDATE]: Term 1 election: Requested vote from peers b1862e4707474c028bacb22100da7091 (127.19.183.193:40595), 49c5be84584b4209b0878618dbfde238 (127.19.183.194:41775)
I20260809 12:21:43.833838 15736 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67cbbdd1aded432d85460413c7d1c868" candidate_uuid: "c40b851c196f4005a4cf03ee41a8dd94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b1862e4707474c028bacb22100da7091"
I20260809 12:21:43.833881 15923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "67cbbdd1aded432d85460413c7d1c868" candidate_uuid: "c40b851c196f4005a4cf03ee41a8dd94" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "49c5be84584b4209b0878618dbfde238"
I20260809 12:21:43.834033 15736 raft_consensus.cc:3037] T 67cbbdd1aded432d85460413c7d1c868 P b1862e4707474c028bacb22100da7091 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:43.834137 15923 raft_consensus.cc:3037] T 67cbbdd1aded432d85460413c7d1c868 P 49c5be84584b4209b0878618dbfde238 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:43.836426 15736 raft_consensus.cc:2454] T 67cbbdd1aded432d85460413c7d1c868 P b1862e4707474c028bacb22100da7091 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40b851c196f4005a4cf03ee41a8dd94 in term 1.
I20260809 12:21:43.836550 15923 raft_consensus.cc:2454] T 67cbbdd1aded432d85460413c7d1c868 P 49c5be84584b4209b0878618dbfde238 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c40b851c196f4005a4cf03ee41a8dd94 in term 1.
I20260809 12:21:43.836850 16018 leader_election.cc:302] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b1862e4707474c028bacb22100da7091, c40b851c196f4005a4cf03ee41a8dd94; no voters:
I20260809 12:21:43.837203 16231 raft_consensus.cc:2781] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:21:43.837682 16231 raft_consensus.cc:686] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94 [term 1 LEADER]: Becoming Leader. State: Replica: c40b851c196f4005a4cf03ee41a8dd94, State: Running, Role: LEADER
I20260809 12:21:43.838308 16231 consensus_queue.cc:237] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1862e4707474c028bacb22100da7091" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40595 } } peers { permanent_uuid: "c40b851c196f4005a4cf03ee41a8dd94" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 39985 } } peers { permanent_uuid: "49c5be84584b4209b0878618dbfde238" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41775 } }
I20260809 12:21:43.843050 15528 catalog_manager.cc:5495] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94 reported cstate change: term changed from 0 to 1, leader changed from <none> to c40b851c196f4005a4cf03ee41a8dd94 (127.19.183.195). New cstate: current_term: 1 leader_uuid: "c40b851c196f4005a4cf03ee41a8dd94" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b1862e4707474c028bacb22100da7091" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40595 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c40b851c196f4005a4cf03ee41a8dd94" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 39985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "49c5be84584b4209b0878618dbfde238" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41775 } health_report { overall_health: UNKNOWN } } }
W20260809 12:21:43.877349 16189 tablet.cc:2369] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:21:43.895038 16215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:43.889741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45800 (local address 127.19.183.195:39985)
0809 12:21:43.889908 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:43.889912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:43.890052 (+ 140us) server_negotiation.cc:407] Connection header received
0809 12:21:43.890225 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:43.890229 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:43.890307 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:43.890432 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:21:43.890820 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.891541 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.892918 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.893332 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.893422 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:43.893936 (+ 514us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:21:43.894418 (+ 482us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:21:43.894548 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:43.894673 (+ 125us) server_negotiation.cc:299] Negotiation successful
0809 12:21:43.894729 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 12:21:43.903530 15736 raft_consensus.cc:1261] T 67cbbdd1aded432d85460413c7d1c868 P b1862e4707474c028bacb22100da7091 [term 1 FOLLOWER]: Refusing update from remote peer c40b851c196f4005a4cf03ee41a8dd94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:21:43.903580 15923 raft_consensus.cc:1261] T 67cbbdd1aded432d85460413c7d1c868 P 49c5be84584b4209b0878618dbfde238 [term 1 FOLLOWER]: Refusing update from remote peer c40b851c196f4005a4cf03ee41a8dd94: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:21:43.904532 16235 consensus_queue.cc:1035] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "49c5be84584b4209b0878618dbfde238" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:21:43.904933 16231 consensus_queue.cc:1035] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b1862e4707474c028bacb22100da7091" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 40595 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 12:21:43.916045 15990 tablet.cc:2369] T 67cbbdd1aded432d85460413c7d1c868 P 49c5be84584b4209b0878618dbfde238: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:21:43.916064 16240 mvcc.cc:205] Tried to move back new op lower bound from 7316595113577861120 to 7316595113326333952. Current Snapshot: MvccSnapshot[applied={T|T < 7316595113577861120}]
I20260809 12:21:43.932351 15527 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:56832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:21:43.932677 15527 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:56832:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:21:43.936353 15527 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f37c53b8c6c4c54b0f2efb54265ba96: Sending DeleteTablet for 3 replicas of tablet 67cbbdd1aded432d85460413c7d1c868
I20260809 12:21:43.937419 16088 tablet_service.cc:1514] Processing DeleteTablet for tablet 67cbbdd1aded432d85460413c7d1c868 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:21:43 UTC) from {username='slave'} at 127.0.0.1:45796
I20260809 12:21:43.937470 15705 tablet_service.cc:1514] Processing DeleteTablet for tablet 67cbbdd1aded432d85460413c7d1c868 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:21:43 UTC) from {username='slave'} at 127.0.0.1:58628
I20260809 12:21:43.938288 16252 tablet_replica.cc:330] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94: stopping tablet replica
I20260809 12:21:43.938759 16252 raft_consensus.cc:2229] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:21:43.939457 16252 raft_consensus.cc:2258] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:21:43.939854 15899 tablet_service.cc:1514] Processing DeleteTablet for tablet 67cbbdd1aded432d85460413c7d1c868 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:21:43 UTC) from {username='slave'} at 127.0.0.1:44848
I20260809 12:21:43.939992 16254 tablet_replica.cc:330] T 67cbbdd1aded432d85460413c7d1c868 P b1862e4707474c028bacb22100da7091: stopping tablet replica
I20260809 12:21:43.940315 16254 raft_consensus.cc:2229] T 67cbbdd1aded432d85460413c7d1c868 P b1862e4707474c028bacb22100da7091 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:21:43.940704 16255 tablet_replica.cc:330] T 67cbbdd1aded432d85460413c7d1c868 P 49c5be84584b4209b0878618dbfde238: stopping tablet replica
I20260809 12:21:43.940826 16254 raft_consensus.cc:2258] T 67cbbdd1aded432d85460413c7d1c868 P b1862e4707474c028bacb22100da7091 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:21:43.941126 16255 raft_consensus.cc:2229] T 67cbbdd1aded432d85460413c7d1c868 P 49c5be84584b4209b0878618dbfde238 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:21:43.941609 16255 raft_consensus.cc:2258] T 67cbbdd1aded432d85460413c7d1c868 P 49c5be84584b4209b0878618dbfde238 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:21:43.941846 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 15614
I20260809 12:21:43.944666 16252 ts_tablet_manager.cc:1905] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:21:43.946189 16254 ts_tablet_manager.cc:1905] T 67cbbdd1aded432d85460413c7d1c868 P b1862e4707474c028bacb22100da7091: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:21:43.946962 16255 ts_tablet_manager.cc:1905] T 67cbbdd1aded432d85460413c7d1c868 P 49c5be84584b4209b0878618dbfde238: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:21:43.954110 16255 ts_tablet_manager.cc:1918] T 67cbbdd1aded432d85460413c7d1c868 P 49c5be84584b4209b0878618dbfde238: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:21:43.954322 16255 log.cc:1196] T 67cbbdd1aded432d85460413c7d1c868 P 49c5be84584b4209b0878618dbfde238: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/67cbbdd1aded432d85460413c7d1c868
I20260809 12:21:43.955011 16255 ts_tablet_manager.cc:1939] T 67cbbdd1aded432d85460413c7d1c868 P 49c5be84584b4209b0878618dbfde238: Deleting consensus metadata
I20260809 12:21:43.957345 15505 catalog_manager.cc:4841] TS 49c5be84584b4209b0878618dbfde238 (127.19.183.194:41775): tablet 67cbbdd1aded432d85460413c7d1c868 (table test-table [id=22f417230ad54908b4d942d15fb69f15]) successfully deleted
I20260809 12:21:43.960350 16252 ts_tablet_manager.cc:1918] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:21:43.960601 16252 log.cc:1196] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/67cbbdd1aded432d85460413c7d1c868
I20260809 12:21:43.961027 16254 ts_tablet_manager.cc:1918] T 67cbbdd1aded432d85460413c7d1c868 P b1862e4707474c028bacb22100da7091: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:21:43.961246 16254 log.cc:1196] T 67cbbdd1aded432d85460413c7d1c868 P b1862e4707474c028bacb22100da7091: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/67cbbdd1aded432d85460413c7d1c868
I20260809 12:21:43.961308 16252 ts_tablet_manager.cc:1939] T 67cbbdd1aded432d85460413c7d1c868 P c40b851c196f4005a4cf03ee41a8dd94: Deleting consensus metadata
I20260809 12:21:43.961851 16254 ts_tablet_manager.cc:1939] T 67cbbdd1aded432d85460413c7d1c868 P b1862e4707474c028bacb22100da7091: Deleting consensus metadata
I20260809 12:21:43.963178 15509 catalog_manager.cc:4841] TS c40b851c196f4005a4cf03ee41a8dd94 (127.19.183.195:39985): tablet 67cbbdd1aded432d85460413c7d1c868 (table test-table [id=22f417230ad54908b4d942d15fb69f15]) successfully deleted
I20260809 12:21:43.963394 16214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:43.942933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58642 (local address 127.19.183.193:40595)
0809 12:21:43.943106 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:43.943114 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:43.955156 (+ 12042us) server_negotiation.cc:407] Connection header received
0809 12:21:43.955444 (+ 288us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:43.955449 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:43.955554 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:43.955631 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:43.956278 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.957341 (+ 1063us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.959264 (+ 1923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:43.959919 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:43.960036 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:43.961591 (+ 1555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:43.961602 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:43.961618 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:43.962414 (+ 796us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:43.962476 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:43.962503 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:43.962594 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:43.962872 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 12:21:43.962991 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 12:21:43.963745 15508 catalog_manager.cc:4841] TS b1862e4707474c028bacb22100da7091 (127.19.183.193:40595): tablet 67cbbdd1aded432d85460413c7d1c868 (table test-table [id=22f417230ad54908b4d942d15fb69f15]) successfully deleted
I20260809 12:21:43.964700 15797 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:44.057477 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 15614
I20260809 12:21:44.082777 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 15810
I20260809 12:21:44.090253 16220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:44.083596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44860 (local address 127.19.183.194:41775)
0809 12:21:44.083797 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:44.083802 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:44.083821 (+ 19us) server_negotiation.cc:407] Connection header received
0809 12:21:44.084003 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:44.084007 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:44.084135 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:44.084227 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:44.084696 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:44.085442 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:44.087206 (+ 1764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:44.087594 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:44.087685 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:44.088818 (+ 1133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:44.088827 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:44.088842 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:44.089472 (+ 630us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:44.089519 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:44.089539 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:44.089635 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:44.089787 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 12:21:44.089873 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260809 12:21:44.090960 15983 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:44.179921 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 15810
I20260809 12:21:44.215054 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 15992
I20260809 12:21:44.223119 16215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:44.215987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45806 (local address 127.19.183.195:39985)
0809 12:21:44.216157 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:44.216162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:44.216228 (+ 66us) server_negotiation.cc:407] Connection header received
0809 12:21:44.216443 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:44.216447 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:44.216525 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:44.216617 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:44.217137 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:44.217906 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:44.219955 (+ 2049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:44.220414 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:44.220503 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:44.221606 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:44.221613 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:44.221637 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:44.222220 (+ 583us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:44.222262 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:44.222281 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:44.222370 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:44.222562 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 12:21:44.222647 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:21:44.223804 16180 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:44.341064 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 15992
I20260809 12:21:44.372781 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 15485
I20260809 12:21:44.373981 15573 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:44.482081 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 15485
2026-08-09T12:21:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (2915 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:21:44 dist-test-slave-39l8 krb5kdc[16267](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:21:44 dist-test-slave-39l8 krb5kdc[16267](info): set up 2 sockets
Aug 09 12:21:44 dist-test-slave-39l8 krb5kdc[16267](info): commencing operation
krb5kdc: starting...
W20260809 12:21:45.796559 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.263s user 0.004s sys 0.037s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:21:45 dist-test-slave-39l8 krb5kdc[16267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:21:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:21:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:21:46.100054 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:36315
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:34757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:36315
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:21:46.438148 16284 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:46.438411 16284 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:46.438566 16284 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:46.445657 16284 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:21:46.445755 16284 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:46.445868 16284 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:46.445955 16284 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:21:46.446014 16284 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:21:46.455236 16284 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:34757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:36315
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:36315
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:46.456441 16284 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:46.457679 16284 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:46.469210 16301 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:46.469154 16295 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:46.469470 16292 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:46.469853 16284 server_base.cc:1034] running on GCE node
I20260809 12:21:46.470568 16284 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:46.471925 16284 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:46.473151 16284 hybrid_clock.cc:648] HybridClock initialized: now 1786278106473110 us; error 64 us; skew 500 ppm
I20260809 12:21:46.473483 16284 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:21:46 dist-test-slave-39l8 krb5kdc[16267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278106, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:21:46.481124 16284 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:21:46.482584 16284 webserver.cc:460] Webserver started at http://127.19.183.254:46673/ using document root <none> and password file <none>
I20260809 12:21:46.483319 16284 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:46.483462 16284 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:46.483765 16284 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:46.486647 16284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "e2e38283e9f24da1ad04524bca12a0a1"
format_stamp: "Formatted at 2026-08-09 12:21:46 on dist-test-slave-39l8"
server_key: "4dc692cb4f78a080b2ca7dfc8769269a"
server_key_iv: "4e3548ca2690735e5925b7c831d0872f"
server_key_version: "encryptionkey@0"
I20260809 12:21:46.487437 16284 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "e2e38283e9f24da1ad04524bca12a0a1"
format_stamp: "Formatted at 2026-08-09 12:21:46 on dist-test-slave-39l8"
server_key: "4dc692cb4f78a080b2ca7dfc8769269a"
server_key_iv: "4e3548ca2690735e5925b7c831d0872f"
server_key_version: "encryptionkey@0"
I20260809 12:21:46.492277 16284 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260809 12:21:46.495590 16311 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:46.496793 16284 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:21:46.496970 16284 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "e2e38283e9f24da1ad04524bca12a0a1"
format_stamp: "Formatted at 2026-08-09 12:21:46 on dist-test-slave-39l8"
server_key: "4dc692cb4f78a080b2ca7dfc8769269a"
server_key_iv: "4e3548ca2690735e5925b7c831d0872f"
server_key_version: "encryptionkey@0"
I20260809 12:21:46.497159 16284 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:46.516271 16284 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:46.520750 16284 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:46.521039 16284 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:46.539429 16284 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:36315
I20260809 12:21:46.539448 16388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:36315 every 8 connection(s)
I20260809 12:21:46.540777 16284 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:21:46.541090 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 16284
I20260809 12:21:46.541270 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:21:46.541782 20191 external_mini_cluster.cc:1428] Setting key 67ecb8e165528aaa98e057d6ad430cb0
I20260809 12:21:46.545176 16390 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 12:21:46 dist-test-slave-39l8 krb5kdc[16267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:21:46.558836 16390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1: Bootstrap starting.
I20260809 12:21:46.561940 16390 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:46.563305 16390 log.cc:824] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:46.566053 16393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:46.543713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46152 (local address 127.19.183.254:36315)
0809 12:21:46.544528 (+ 815us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:46.544540 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:46.544580 (+ 40us) server_negotiation.cc:407] Connection header received
0809 12:21:46.546344 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:46.546366 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:46.546792 (+ 426us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:46.547121 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:46.548986 (+ 1865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:46.550294 (+ 1308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:46.552191 (+ 1897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:46.552844 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:46.552972 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:46.558275 (+ 5303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:46.558282 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:46.558295 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:46.558333 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:46.561295 (+ 2962us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:46.562165 (+ 870us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:46.562173 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:46.562181 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:46.562429 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:46.562911 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:46.562916 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:46.562920 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:46.563472 (+ 552us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:46.563780 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:46.564289 (+ 509us) server_negotiation.cc:299] Negotiation successful
0809 12:21:46.564754 (+ 465us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":592,"thread_start_us":357,"threads_started":1}
I20260809 12:21:46.566663 16390 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1: No bootstrap required, opened a new log
I20260809 12:21:46.573910 16390 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e38283e9f24da1ad04524bca12a0a1" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36315 } }
I20260809 12:21:46.574258 16390 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:46.574354 16390 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2e38283e9f24da1ad04524bca12a0a1, State: Initialized, Role: FOLLOWER
I20260809 12:21:46.574997 16390 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e38283e9f24da1ad04524bca12a0a1" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36315 } }
I20260809 12:21:46.575275 16390 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:21:46.575424 16390 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:21:46.575593 16390 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:46.577648 16390 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e38283e9f24da1ad04524bca12a0a1" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36315 } }
I20260809 12:21:46.578115 16390 leader_election.cc:302] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2e38283e9f24da1ad04524bca12a0a1; no voters:
I20260809 12:21:46.578667 16390 leader_election.cc:288] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:21:46.578845 16397 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:21:46.579519 16397 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1 [term 1 LEADER]: Becoming Leader. State: Replica: e2e38283e9f24da1ad04524bca12a0a1, State: Running, Role: LEADER
I20260809 12:21:46.580065 16397 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e38283e9f24da1ad04524bca12a0a1" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36315 } }
I20260809 12:21:46.580762 16390 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:21:46.583011 16398 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2e38283e9f24da1ad04524bca12a0a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e38283e9f24da1ad04524bca12a0a1" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36315 } } }
I20260809 12:21:46.583501 16398 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1 [sys.catalog]: This master's current role is: LEADER
I20260809 12:21:46.583324 16399 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2e38283e9f24da1ad04524bca12a0a1. Latest consensus state: current_term: 1 leader_uuid: "e2e38283e9f24da1ad04524bca12a0a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2e38283e9f24da1ad04524bca12a0a1" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36315 } } }
I20260809 12:21:46.583603 16399 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1 [sys.catalog]: This master's current role is: LEADER
I20260809 12:21:46.590698 16405 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:21:46.596812 16405 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:21:46.606218 16405 catalog_manager.cc:1347] Generated new cluster ID: 86c8a4b7767f4b1aa92f9e6564558ccf
I20260809 12:21:46.606315 16405 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:21:46.624547 16405 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:21:46.625768 16405 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:21:46.638926 16405 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1: Generated new TSK 0
I20260809 12:21:46.639671 16405 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.19.183.193@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.193@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
Entry for principal kudu/127.19.183.193 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab.
I20260809 12:21:46.715832 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:36315
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:34757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:21:47.060910 16424 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:47.061231 16424 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:47.061404 16424 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:47.068913 16424 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:47.069077 16424 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:47.069245 16424 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:21:47.078900 16424 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:34757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.193.keytab
--principal=kudu/127.19.183.193
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:36315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:47.080163 16424 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:47.081585 16424 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:47.095119 16434 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:47.095148 16429 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:47.095152 16432 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:47.095983 16424 server_base.cc:1034] running on GCE node
I20260809 12:21:47.096577 16424 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:47.097478 16424 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:47.098670 16424 hybrid_clock.cc:648] HybridClock initialized: now 1786278107098639 us; error 25 us; skew 500 ppm
I20260809 12:21:47.098977 16424 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:21:47 dist-test-slave-39l8 krb5kdc[16267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278107, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:21:47.102272 16424 init.cc:377] Logged in from keytab as kudu/127.19.183.193@KRBTEST.COM (short username kudu)
I20260809 12:21:47.103680 16424 webserver.cc:460] Webserver started at http://127.19.183.193:40459/ using document root <none> and password file <none>
I20260809 12:21:47.104389 16424 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:47.104522 16424 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:47.104833 16424 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:47.107709 16424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "4a32c96d30a94b358fc2f3708f05bc5d"
format_stamp: "Formatted at 2026-08-09 12:21:47 on dist-test-slave-39l8"
server_key: "82deedbe9f23c9b5ec18518045a4877b"
server_key_iv: "00cab753d0e32cb7f577763af06a5534"
server_key_version: "encryptionkey@0"
I20260809 12:21:47.108475 16424 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "4a32c96d30a94b358fc2f3708f05bc5d"
format_stamp: "Formatted at 2026-08-09 12:21:47 on dist-test-slave-39l8"
server_key: "82deedbe9f23c9b5ec18518045a4877b"
server_key_iv: "00cab753d0e32cb7f577763af06a5534"
server_key_version: "encryptionkey@0"
I20260809 12:21:47.113869 16424 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260809 12:21:47.117205 16443 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:47.118446 16424 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:21:47.118618 16424 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "4a32c96d30a94b358fc2f3708f05bc5d"
format_stamp: "Formatted at 2026-08-09 12:21:47 on dist-test-slave-39l8"
server_key: "82deedbe9f23c9b5ec18518045a4877b"
server_key_iv: "00cab753d0e32cb7f577763af06a5534"
server_key_version: "encryptionkey@0"
I20260809 12:21:47.118857 16424 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:47.134378 16424 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:47.138211 16424 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:47.138532 16424 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:47.139549 16424 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:21:47.140807 16424 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:21:47.140900 16424 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:47.140992 16424 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:21:47.141036 16424 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:47.174021 16424 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:37595
I20260809 12:21:47.174044 16609 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:37595 every 8 connection(s)
I20260809 12:21:47.175423 16424 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:21:47.178689 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 16424
I20260809 12:21:47.178934 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:21:47.179447 20191 external_mini_cluster.cc:1428] Setting key a8f4c794b509e39fc6327baa6f8ead51
Aug 09 12:21:47 dist-test-slave-39l8 krb5kdc[16267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278107, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.193@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:21:47.195583 16614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:47.177943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:48701 (local address 127.19.183.254:36315)
0809 12:21:47.178403 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:47.178407 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:47.179414 (+ 1007us) server_negotiation.cc:407] Connection header received
0809 12:21:47.181472 (+ 2058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:47.181478 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:47.181564 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:47.181678 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:47.183294 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:47.184015 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:47.185596 (+ 1581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:47.186005 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:47.186156 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:47.189712 (+ 3556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:47.189717 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:47.189720 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:47.189752 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:47.192441 (+ 2689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:47.193167 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:47.193172 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:47.193193 (+ 21us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:47.193357 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:47.193803 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:47.193806 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:47.193809 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:47.194070 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:47.194239 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:47.194987 (+ 748us) server_negotiation.cc:299] Negotiation successful
0809 12:21:47.195219 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":258,"threads_started":1}
I20260809 12:21:47.196740 16617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:47.178397 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:36315 (local address 127.19.183.193:48701)
0809 12:21:47.179257 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:47.179296 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:47.181222 (+ 1926us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:47.181909 (+ 687us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:47.181917 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:47.182500 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:21:47.183125 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:47.183142 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:47.184208 (+ 1066us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:47.184212 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:47.185421 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:47.185434 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:47.186250 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:47.186260 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:47.186494 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:47.187189 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:21:47.187216 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:21:47.189493 (+ 2277us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:21:47.192584 (+ 3091us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:21:47.192601 (+ 17us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:21:47.192616 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:21:47.193025 (+ 409us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:21:47.193470 (+ 445us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:21:47.193475 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:21:47.193479 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:21:47.193675 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:21:47.194212 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:21:47.194220 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:21:47.194563 (+ 343us) client_negotiation.cc:769] Sending connection context
0809 12:21:47.194872 (+ 309us) client_negotiation.cc:240] Negotiation successful
0809 12:21:47.195369 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"thread_start_us":381,"threads_started":1}
I20260809 12:21:47.198274 16610 heartbeater.cc:347] Connected to a master server at 127.19.183.254:36315
I20260809 12:21:47.198576 16610 heartbeater.cc:464] Registering TS with master...
I20260809 12:21:47.199462 16610 heartbeater.cc:511] Master 127.19.183.254:36315 requested a full tablet report, sending...
I20260809 12:21:47.201930 16339 ts_manager.cc:194] Registered new tserver with Master: 4a32c96d30a94b358fc2f3708f05bc5d (127.19.183.193:37595)
I20260809 12:21:47.203444 16339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.193@KRBTEST.COM'} at 127.19.183.193:48701
WARNING: no policy specified for kudu/127.19.183.194@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.194@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
Entry for principal kudu/127.19.183.194 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab.
I20260809 12:21:47.254061 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:36315
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:34757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:21:47.604897 16623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:47.605237 16623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:47.605410 16623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:47.612831 16623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:47.612988 16623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:47.613173 16623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:21:47.623520 16623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:34757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.194.keytab
--principal=kudu/127.19.183.194
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:36315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:47.624913 16623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:47.626250 16623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:47.639565 16630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:47.639559 16631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:47.640017 16636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:47.640543 16623 server_base.cc:1034] running on GCE node
I20260809 12:21:47.641194 16623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:47.642163 16623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:47.643388 16623 hybrid_clock.cc:648] HybridClock initialized: now 1786278107643350 us; error 49 us; skew 500 ppm
I20260809 12:21:47.643719 16623 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:21:47 dist-test-slave-39l8 krb5kdc[16267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278107, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:21:47.647055 16623 init.cc:377] Logged in from keytab as kudu/127.19.183.194@KRBTEST.COM (short username kudu)
I20260809 12:21:47.648499 16623 webserver.cc:460] Webserver started at http://127.19.183.194:42065/ using document root <none> and password file <none>
I20260809 12:21:47.649174 16623 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:47.649277 16623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:47.649561 16623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:47.652560 16623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "4108675c9b884f1cb4f9f052e9bef752"
format_stamp: "Formatted at 2026-08-09 12:21:47 on dist-test-slave-39l8"
server_key: "3e2c65cad1cd49f64a0b51409b11794e"
server_key_iv: "f87a3a637a3c2e8ed09820d9f9a75b12"
server_key_version: "encryptionkey@0"
I20260809 12:21:47.653321 16623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "4108675c9b884f1cb4f9f052e9bef752"
format_stamp: "Formatted at 2026-08-09 12:21:47 on dist-test-slave-39l8"
server_key: "3e2c65cad1cd49f64a0b51409b11794e"
server_key_iv: "f87a3a637a3c2e8ed09820d9f9a75b12"
server_key_version: "encryptionkey@0"
I20260809 12:21:47.658690 16623 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.005s
I20260809 12:21:47.662252 16646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:47.663683 16623 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260809 12:21:47.663821 16623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "4108675c9b884f1cb4f9f052e9bef752"
format_stamp: "Formatted at 2026-08-09 12:21:47 on dist-test-slave-39l8"
server_key: "3e2c65cad1cd49f64a0b51409b11794e"
server_key_iv: "f87a3a637a3c2e8ed09820d9f9a75b12"
server_key_version: "encryptionkey@0"
I20260809 12:21:47.663990 16623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:47.685350 16623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:47.689826 16623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:47.690171 16623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:47.691277 16623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:21:47.692678 16623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:21:47.692787 16623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:47.692942 16623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:21:47.693017 16623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:47.727048 16623 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:33591
I20260809 12:21:47.727111 16815 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:33591 every 8 connection(s)
I20260809 12:21:47.728343 16623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:21:47.736196 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 16623
I20260809 12:21:47.736388 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:21:47.736840 20191 external_mini_cluster.cc:1428] Setting key 14064fe0fbe763dc60217b6ab13b5364
Aug 09 12:21:47 dist-test-slave-39l8 krb5kdc[16267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278107, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.194@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:21:47.748570 16820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:47.730912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:50105 (local address 127.19.183.254:36315)
0809 12:21:47.731423 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:47.731427 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:47.732246 (+ 819us) server_negotiation.cc:407] Connection header received
0809 12:21:47.734125 (+ 1879us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:47.734131 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:47.734226 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:47.734347 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:47.735885 (+ 1538us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:47.736695 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:47.738212 (+ 1517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:47.738678 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:47.738844 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:47.742761 (+ 3917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:47.742767 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:47.742770 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:47.742803 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:47.745283 (+ 2480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:47.746237 (+ 954us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:47.746242 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:47.746244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:47.746377 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:47.746885 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:47.746888 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:47.746891 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:47.747167 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:47.747278 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:47.748146 (+ 868us) server_negotiation.cc:299] Negotiation successful
0809 12:21:47.748305 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":400,"thread_start_us":279,"threads_started":1}
I20260809 12:21:47.750015 16821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:47.731352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:36315 (local address 127.19.183.194:50105)
0809 12:21:47.732067 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:47.732120 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:47.733854 (+ 1734us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:47.734582 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:47.734591 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:47.735162 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:21:47.735717 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:47.735735 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:47.736844 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:47.736848 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:47.738056 (+ 1208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:47.738069 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:47.738950 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:47.738957 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:47.739183 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:47.739864 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:21:47.739890 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:21:47.742494 (+ 2604us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:21:47.745436 (+ 2942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:21:47.745447 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:21:47.745463 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:21:47.746081 (+ 618us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:21:47.746492 (+ 411us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:21:47.746499 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:21:47.746505 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:21:47.746776 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:21:47.747287 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:21:47.747297 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:21:47.747728 (+ 431us) client_negotiation.cc:769] Sending connection context
0809 12:21:47.748023 (+ 295us) client_negotiation.cc:240] Negotiation successful
0809 12:21:47.748594 (+ 571us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":277,"threads_started":1}
I20260809 12:21:47.751545 16816 heartbeater.cc:347] Connected to a master server at 127.19.183.254:36315
I20260809 12:21:47.751852 16816 heartbeater.cc:464] Registering TS with master...
I20260809 12:21:47.752558 16816 heartbeater.cc:511] Master 127.19.183.254:36315 requested a full tablet report, sending...
I20260809 12:21:47.754011 16339 ts_manager.cc:194] Registered new tserver with Master: 4108675c9b884f1cb4f9f052e9bef752 (127.19.183.194:33591)
I20260809 12:21:47.755327 16339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.194@KRBTEST.COM'} at 127.19.183.194:50105
WARNING: no policy specified for kudu/127.19.183.195@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.195@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
Entry for principal kudu/127.19.183.195 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab.
I20260809 12:21:47.810012 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:36315
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:34757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:21:48.157500 16824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:48.157815 16824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:48.158035 16824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:48.165912 16824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:48.166049 16824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:48.166210 16824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:21:48.176183 16824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:34757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.195.keytab
--principal=kudu/127.19.183.195
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:36315
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:48.177456 16824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:48.178830 16824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:48.192202 16831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:48.192256 16832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:48.192312 16834 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:48.194020 16824 server_base.cc:1034] running on GCE node
I20260809 12:21:48.194643 16824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:48.195640 16824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:48.196854 16824 hybrid_clock.cc:648] HybridClock initialized: now 1786278108196805 us; error 60 us; skew 500 ppm
I20260809 12:21:48.197173 16824 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:21:48 dist-test-slave-39l8 krb5kdc[16267](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278108, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:21:48.200476 16824 init.cc:377] Logged in from keytab as kudu/127.19.183.195@KRBTEST.COM (short username kudu)
I20260809 12:21:48.202028 16824 webserver.cc:460] Webserver started at http://127.19.183.195:35205/ using document root <none> and password file <none>
I20260809 12:21:48.202790 16824 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:48.202898 16824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:48.203166 16824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:48.206066 16824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "eec60939304442ebae620711e9047057"
format_stamp: "Formatted at 2026-08-09 12:21:48 on dist-test-slave-39l8"
server_key: "41ad378e0f7c62ac8101979383955c2d"
server_key_iv: "10021fcd1e775514709ee19742e1eb92"
server_key_version: "encryptionkey@0"
I20260809 12:21:48.206923 16824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "eec60939304442ebae620711e9047057"
format_stamp: "Formatted at 2026-08-09 12:21:48 on dist-test-slave-39l8"
server_key: "41ad378e0f7c62ac8101979383955c2d"
server_key_iv: "10021fcd1e775514709ee19742e1eb92"
server_key_version: "encryptionkey@0"
I20260809 12:21:48.206982 16610 heartbeater.cc:503] Master 127.19.183.254:36315 was elected leader, sending a full tablet report...
I20260809 12:21:48.212472 16824 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:21:48.215916 16844 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:48.217181 16824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:21:48.217456 16824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "eec60939304442ebae620711e9047057"
format_stamp: "Formatted at 2026-08-09 12:21:48 on dist-test-slave-39l8"
server_key: "41ad378e0f7c62ac8101979383955c2d"
server_key_iv: "10021fcd1e775514709ee19742e1eb92"
server_key_version: "encryptionkey@0"
I20260809 12:21:48.217664 16824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:48.239427 16824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:48.243780 16824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:48.244069 16824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:48.245049 16824 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:21:48.246371 16824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:21:48.246443 16824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:48.246603 16824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:21:48.246654 16824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:48.280329 16824 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:35639
I20260809 12:21:48.280397 17007 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:35639 every 8 connection(s)
I20260809 12:21:48.281765 16824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:21:48.283958 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 16824
I20260809 12:21:48.284155 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:21:48.284771 20191 external_mini_cluster.cc:1428] Setting key 6b871da425564886ab2bbdb9a9bf7607
Aug 09 12:21:48 dist-test-slave-39l8 krb5kdc[16267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278108, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.195@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:21:48.302202 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:48.284490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:38175 (local address 127.19.183.254:36315)
0809 12:21:48.285033 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:48.285037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:48.286081 (+ 1044us) server_negotiation.cc:407] Connection header received
0809 12:21:48.288277 (+ 2196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:48.288281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:48.288348 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:48.288438 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:48.289893 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.290682 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.292315 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.292704 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.292848 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:48.296141 (+ 3293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:48.296146 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:48.296149 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:48.296180 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:48.299009 (+ 2829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:48.299845 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:48.299850 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:48.299853 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:48.300001 (+ 148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:48.300456 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:48.300461 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:48.300465 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:48.300776 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:48.300943 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:48.301601 (+ 658us) server_negotiation.cc:299] Negotiation successful
0809 12:21:48.301817 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":440,"thread_start_us":262,"threads_started":1}
I20260809 12:21:48.303432 17012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:48.285049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:36315 (local address 127.19.183.195:38175)
0809 12:21:48.285929 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:48.285983 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:48.288025 (+ 2042us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:48.288652 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:48.288660 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:48.289188 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:21:48.289726 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:48.289742 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.290971 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.290975 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:48.292185 (+ 1210us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:48.292203 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.292970 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.292976 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:48.293187 (+ 211us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:48.293791 (+ 604us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:21:48.293813 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:21:48.295944 (+ 2131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:21:48.299140 (+ 3196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:21:48.299149 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:21:48.299163 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:21:48.299710 (+ 547us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:21:48.300132 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:21:48.300140 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:21:48.300144 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:21:48.300334 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:21:48.300908 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:21:48.300916 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:21:48.301233 (+ 317us) client_negotiation.cc:769] Sending connection context
0809 12:21:48.301488 (+ 255us) client_negotiation.cc:240] Negotiation successful
0809 12:21:48.302015 (+ 527us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":613,"thread_start_us":378,"threads_started":1}
I20260809 12:21:48.304795 17009 heartbeater.cc:347] Connected to a master server at 127.19.183.254:36315
I20260809 12:21:48.305055 17009 heartbeater.cc:464] Registering TS with master...
I20260809 12:21:48.305699 17009 heartbeater.cc:511] Master 127.19.183.254:36315 requested a full tablet report, sending...
I20260809 12:21:48.307044 16339 ts_manager.cc:194] Registered new tserver with Master: eec60939304442ebae620711e9047057 (127.19.183.195:35639)
I20260809 12:21:48.307809 16339 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.19.183.195@KRBTEST.COM'} at 127.19.183.195:38175
I20260809 12:21:48.312088 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:21:48.328243 17011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:48.316034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46162 (local address 127.19.183.254:36315)
0809 12:21:48.316258 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:48.316267 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:48.316588 (+ 321us) server_negotiation.cc:407] Connection header received
0809 12:21:48.316715 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:48.316719 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:48.316777 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:48.316854 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:48.318180 (+ 1326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.318914 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.320179 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.320548 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.320639 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:48.323273 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:48.323281 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:48.323285 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:48.323322 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:48.325892 (+ 2570us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:48.326475 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:48.326479 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:48.326481 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:48.326596 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:48.326981 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:48.326984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:48.326986 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:48.327290 (+ 304us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:48.327468 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:48.327743 (+ 275us) server_negotiation.cc:299] Negotiation successful
0809 12:21:48.327900 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 12:21:48.331686 16339 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46162:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:21:48.338526 16339 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:21:48.360844 17030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:48.350645 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:35639 (local address 127.0.0.1:33360)
0809 12:21:48.352095 (+ 1450us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:48.352113 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:48.352231 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:48.353244 (+ 1013us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:48.353252 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:48.353309 (+ 57us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:48.354533 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:48.354549 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.356310 (+ 1761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.356317 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:48.358436 (+ 2119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:48.358446 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.359770 (+ 1324us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.359776 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:48.359972 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:48.360038 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 12:21:48.360167 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 12:21:48.360340 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1318,"mutex_wait_us":46,"spinlock_wait_cycles":9856,"thread_start_us":1173,"threads_started":1}
I20260809 12:21:48.361130 17031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:48.350619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35286 (local address 127.19.183.193:37595)
0809 12:21:48.352660 (+ 2041us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:48.352668 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:48.352695 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:21:48.352788 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:48.352797 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:48.353008 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:48.353454 (+ 446us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:48.354080 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.355469 (+ 1389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.358379 (+ 2910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.359901 (+ 1522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.360093 (+ 192us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:48.360572 (+ 479us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:48.360645 (+ 73us) server_negotiation.cc:299] Negotiation successful
0809 12:21:48.360802 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1881,"thread_start_us":1736,"threads_started":1}
I20260809 12:21:48.360884 17028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:48.350454 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:37595 (local address 127.0.0.1:35286)
0809 12:21:48.351284 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:48.351319 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:48.351507 (+ 188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:48.353244 (+ 1737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:48.353252 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:48.353308 (+ 56us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:48.353919 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:48.353936 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.355647 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.355652 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:48.358205 (+ 2553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:48.358225 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.360116 (+ 1891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.360125 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:48.360285 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:48.360316 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 12:21:48.360385 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 12:21:48.360503 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":569,"mutex_wait_us":3,"spinlock_wait_cycles":7296,"thread_start_us":542,"threads_started":1}
I20260809 12:21:48.361503 17032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:48.351286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33360 (local address 127.19.183.195:35639)
0809 12:21:48.352135 (+ 849us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:48.352146 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:48.352182 (+ 36us) server_negotiation.cc:407] Connection header received
0809 12:21:48.352441 (+ 259us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:48.352449 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:48.352714 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:48.352897 (+ 183us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:48.354801 (+ 1904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.356171 (+ 1370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.358573 (+ 2402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.359561 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.359769 (+ 208us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:48.359947 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:48.360989 (+ 1042us) server_negotiation.cc:299] Negotiation successful
0809 12:21:48.361157 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":453,"threads_started":1}
I20260809 12:21:48.362594 17033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:48.352202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:33591 (local address 127.0.0.1:44840)
0809 12:21:48.354541 (+ 2339us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:48.354561 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:48.354681 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:48.355467 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:48.355472 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:48.355498 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:48.355972 (+ 474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:48.355983 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.357727 (+ 1744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.357732 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:48.359759 (+ 2027us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:48.359775 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.361832 (+ 2057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.361838 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:48.362019 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:48.362077 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 12:21:48.362166 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 12:21:48.362290 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2193,"thread_start_us":2009,"threads_started":1}
I20260809 12:21:48.363380 17034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:48.352788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44840 (local address 127.19.183.194:33591)
0809 12:21:48.354886 (+ 2098us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:48.354896 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:48.354925 (+ 29us) server_negotiation.cc:407] Connection header received
0809 12:21:48.355021 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:48.355029 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:48.355247 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:48.355449 (+ 202us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:48.356164 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.357544 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.359964 (+ 2420us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.361340 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.361505 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:48.361685 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:48.362853 (+ 1168us) server_negotiation.cc:299] Negotiation successful
0809 12:21:48.363047 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1180,"thread_start_us":1012,"threads_started":1}
I20260809 12:21:48.365538 16512 tablet_service.cc:1467] Processing CreateTablet for tablet 58f9e874159f4cac9a967a487c5d4acc (DEFAULT_TABLE table=test-table [id=9209a81e96da411398030da65c8ce388]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:21:48.365500 16911 tablet_service.cc:1467] Processing CreateTablet for tablet 58f9e874159f4cac9a967a487c5d4acc (DEFAULT_TABLE table=test-table [id=9209a81e96da411398030da65c8ce388]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:21:48.367000 16911 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58f9e874159f4cac9a967a487c5d4acc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:48.367101 16512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58f9e874159f4cac9a967a487c5d4acc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:48.366811 16716 tablet_service.cc:1467] Processing CreateTablet for tablet 58f9e874159f4cac9a967a487c5d4acc (DEFAULT_TABLE table=test-table [id=9209a81e96da411398030da65c8ce388]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:21:48.368193 16716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 58f9e874159f4cac9a967a487c5d4acc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:48.378430 17035 tablet_bootstrap.cc:492] T 58f9e874159f4cac9a967a487c5d4acc P eec60939304442ebae620711e9047057: Bootstrap starting.
I20260809 12:21:48.381426 17038 tablet_bootstrap.cc:492] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d: Bootstrap starting.
I20260809 12:21:48.382375 17035 tablet_bootstrap.cc:654] T 58f9e874159f4cac9a967a487c5d4acc P eec60939304442ebae620711e9047057: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:48.383067 17039 tablet_bootstrap.cc:492] T 58f9e874159f4cac9a967a487c5d4acc P 4108675c9b884f1cb4f9f052e9bef752: Bootstrap starting.
I20260809 12:21:48.384680 17035 log.cc:824] T 58f9e874159f4cac9a967a487c5d4acc P eec60939304442ebae620711e9047057: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:48.385056 17038 tablet_bootstrap.cc:654] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:48.386261 17038 log.cc:824] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:48.386857 17039 tablet_bootstrap.cc:654] T 58f9e874159f4cac9a967a487c5d4acc P 4108675c9b884f1cb4f9f052e9bef752: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:48.387996 17039 log.cc:824] T 58f9e874159f4cac9a967a487c5d4acc P 4108675c9b884f1cb4f9f052e9bef752: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:48.389236 17035 tablet_bootstrap.cc:492] T 58f9e874159f4cac9a967a487c5d4acc P eec60939304442ebae620711e9047057: No bootstrap required, opened a new log
I20260809 12:21:48.389236 17038 tablet_bootstrap.cc:492] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d: No bootstrap required, opened a new log
I20260809 12:21:48.389523 17038 ts_tablet_manager.cc:1397] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d: Time spent bootstrapping tablet: real 0.008s user 0.003s sys 0.003s
I20260809 12:21:48.389523 17035 ts_tablet_manager.cc:1397] T 58f9e874159f4cac9a967a487c5d4acc P eec60939304442ebae620711e9047057: Time spent bootstrapping tablet: real 0.011s user 0.007s sys 0.000s
I20260809 12:21:48.390535 17039 tablet_bootstrap.cc:492] T 58f9e874159f4cac9a967a487c5d4acc P 4108675c9b884f1cb4f9f052e9bef752: No bootstrap required, opened a new log
I20260809 12:21:48.390826 17039 ts_tablet_manager.cc:1397] T 58f9e874159f4cac9a967a487c5d4acc P 4108675c9b884f1cb4f9f052e9bef752: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.003s
I20260809 12:21:48.398077 17039 raft_consensus.cc:348] T 58f9e874159f4cac9a967a487c5d4acc P 4108675c9b884f1cb4f9f052e9bef752 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a32c96d30a94b358fc2f3708f05bc5d" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37595 } } peers { permanent_uuid: "eec60939304442ebae620711e9047057" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35639 } } peers { permanent_uuid: "4108675c9b884f1cb4f9f052e9bef752" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 33591 } }
I20260809 12:21:48.398447 17039 raft_consensus.cc:374] T 58f9e874159f4cac9a967a487c5d4acc P 4108675c9b884f1cb4f9f052e9bef752 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:48.398546 17039 raft_consensus.cc:729] T 58f9e874159f4cac9a967a487c5d4acc P 4108675c9b884f1cb4f9f052e9bef752 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4108675c9b884f1cb4f9f052e9bef752, State: Initialized, Role: FOLLOWER
I20260809 12:21:48.399178 17039 consensus_queue.cc:260] T 58f9e874159f4cac9a967a487c5d4acc P 4108675c9b884f1cb4f9f052e9bef752 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a32c96d30a94b358fc2f3708f05bc5d" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37595 } } peers { permanent_uuid: "eec60939304442ebae620711e9047057" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35639 } } peers { permanent_uuid: "4108675c9b884f1cb4f9f052e9bef752" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 33591 } }
I20260809 12:21:48.400270 16816 heartbeater.cc:503] Master 127.19.183.254:36315 was elected leader, sending a full tablet report...
I20260809 12:21:48.400578 17035 raft_consensus.cc:348] T 58f9e874159f4cac9a967a487c5d4acc P eec60939304442ebae620711e9047057 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a32c96d30a94b358fc2f3708f05bc5d" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37595 } } peers { permanent_uuid: "eec60939304442ebae620711e9047057" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35639 } } peers { permanent_uuid: "4108675c9b884f1cb4f9f052e9bef752" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 33591 } }
I20260809 12:21:48.400578 17038 raft_consensus.cc:348] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a32c96d30a94b358fc2f3708f05bc5d" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37595 } } peers { permanent_uuid: "eec60939304442ebae620711e9047057" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35639 } } peers { permanent_uuid: "4108675c9b884f1cb4f9f052e9bef752" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 33591 } }
I20260809 12:21:48.400997 17038 raft_consensus.cc:374] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:48.400985 17035 raft_consensus.cc:374] T 58f9e874159f4cac9a967a487c5d4acc P eec60939304442ebae620711e9047057 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:48.401090 17038 raft_consensus.cc:729] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a32c96d30a94b358fc2f3708f05bc5d, State: Initialized, Role: FOLLOWER
I20260809 12:21:48.401084 17035 raft_consensus.cc:729] T 58f9e874159f4cac9a967a487c5d4acc P eec60939304442ebae620711e9047057 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eec60939304442ebae620711e9047057, State: Initialized, Role: FOLLOWER
I20260809 12:21:48.401199 17039 ts_tablet_manager.cc:1428] T 58f9e874159f4cac9a967a487c5d4acc P 4108675c9b884f1cb4f9f052e9bef752: Time spent starting tablet: real 0.010s user 0.009s sys 0.001s
I20260809 12:21:48.401671 17038 consensus_queue.cc:260] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a32c96d30a94b358fc2f3708f05bc5d" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37595 } } peers { permanent_uuid: "eec60939304442ebae620711e9047057" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35639 } } peers { permanent_uuid: "4108675c9b884f1cb4f9f052e9bef752" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 33591 } }
I20260809 12:21:48.401808 17035 consensus_queue.cc:260] T 58f9e874159f4cac9a967a487c5d4acc P eec60939304442ebae620711e9047057 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a32c96d30a94b358fc2f3708f05bc5d" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37595 } } peers { permanent_uuid: "eec60939304442ebae620711e9047057" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35639 } } peers { permanent_uuid: "4108675c9b884f1cb4f9f052e9bef752" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 33591 } }
I20260809 12:21:48.403731 17038 ts_tablet_manager.cc:1428] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d: Time spent starting tablet: real 0.014s user 0.014s sys 0.001s
I20260809 12:21:48.405941 17009 heartbeater.cc:503] Master 127.19.183.254:36315 was elected leader, sending a full tablet report...
I20260809 12:21:48.407202 17035 ts_tablet_manager.cc:1428] T 58f9e874159f4cac9a967a487c5d4acc P eec60939304442ebae620711e9047057: Time spent starting tablet: real 0.018s user 0.015s sys 0.004s
W20260809 12:21:48.427788 16612 tablet.cc:2369] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:21:48.480343 16819 tablet.cc:2369] T 58f9e874159f4cac9a967a487c5d4acc P 4108675c9b884f1cb4f9f052e9bef752: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 12:21:48.533548 17010 tablet.cc:2369] T 58f9e874159f4cac9a967a487c5d4acc P eec60939304442ebae620711e9047057: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:21:48.644762 17044 raft_consensus.cc:482] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:21:48.645125 17044 raft_consensus.cc:504] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a32c96d30a94b358fc2f3708f05bc5d" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37595 } } peers { permanent_uuid: "eec60939304442ebae620711e9047057" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35639 } } peers { permanent_uuid: "4108675c9b884f1cb4f9f052e9bef752" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 33591 } }
I20260809 12:21:48.647241 17044 leader_election.cc:288] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eec60939304442ebae620711e9047057 (127.19.183.195:35639), 4108675c9b884f1cb4f9f052e9bef752 (127.19.183.194:33591)
I20260809 12:21:48.655192 17034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:48.648997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:56543 (local address 127.19.183.194:33591)
0809 12:21:48.649200 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:48.649208 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:48.649229 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:21:48.649465 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:48.649470 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:48.649544 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:48.649607 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:48.650599 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.651401 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.653465 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.654209 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.654412 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:48.654488 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:48.654793 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 12:21:48.654923 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:21:48.655318 17047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:48.648515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:33591 (local address 127.19.183.193:56543)
0809 12:21:48.649153 (+ 638us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:48.649170 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:48.649294 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:48.649827 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:48.649831 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:48.649859 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:48.650399 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:48.650415 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.651570 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.651574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:48.653313 (+ 1739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:48.653326 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.654439 (+ 1113us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.654445 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:48.654610 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:48.654634 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 12:21:48.654754 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 12:21:48.654890 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":28,"thread_start_us":2132,"threads_started":1}
I20260809 12:21:48.655704 17032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:48.647779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:37771 (local address 127.19.183.195:35639)
0809 12:21:48.648028 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:48.648037 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:48.648456 (+ 419us) server_negotiation.cc:407] Connection header received
0809 12:21:48.648713 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:48.648719 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:48.648900 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:48.649078 (+ 178us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:48.650455 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.651756 (+ 1301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.653903 (+ 2147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.654687 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.654894 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:48.654977 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:48.655295 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 12:21:48.655428 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 12:21:48.656078 17046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:48.647491 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:35639 (local address 127.19.183.193:37771)
0809 12:21:48.648326 (+ 835us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:48.648346 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:48.648497 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:48.649443 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:48.649450 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:48.649519 (+ 69us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:48.650205 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:48.650225 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.651971 (+ 1746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.651979 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:48.653743 (+ 1764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:48.653756 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.654947 (+ 1191us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.654954 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:48.655131 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:48.655155 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 12:21:48.655598 (+ 443us) client_negotiation.cc:240] Negotiation successful
0809 12:21:48.655730 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":707,"mutex_wait_us":3,"thread_start_us":558,"threads_started":1}
I20260809 12:21:48.656317 16747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58f9e874159f4cac9a967a487c5d4acc" candidate_uuid: "4a32c96d30a94b358fc2f3708f05bc5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4108675c9b884f1cb4f9f052e9bef752" is_pre_election: true
I20260809 12:21:48.656891 16747 raft_consensus.cc:2454] T 58f9e874159f4cac9a967a487c5d4acc P 4108675c9b884f1cb4f9f052e9bef752 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a32c96d30a94b358fc2f3708f05bc5d in term 0.
I20260809 12:21:48.657009 16937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58f9e874159f4cac9a967a487c5d4acc" candidate_uuid: "4a32c96d30a94b358fc2f3708f05bc5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eec60939304442ebae620711e9047057" is_pre_election: true
I20260809 12:21:48.657761 16937 raft_consensus.cc:2454] T 58f9e874159f4cac9a967a487c5d4acc P eec60939304442ebae620711e9047057 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a32c96d30a94b358fc2f3708f05bc5d in term 0.
I20260809 12:21:48.657779 16447 leader_election.cc:302] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4108675c9b884f1cb4f9f052e9bef752, 4a32c96d30a94b358fc2f3708f05bc5d; no voters:
I20260809 12:21:48.658214 17044 raft_consensus.cc:2781] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:21:48.658375 17044 raft_consensus.cc:482] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:21:48.658499 17044 raft_consensus.cc:3037] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:48.660765 17044 raft_consensus.cc:504] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a32c96d30a94b358fc2f3708f05bc5d" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37595 } } peers { permanent_uuid: "eec60939304442ebae620711e9047057" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35639 } } peers { permanent_uuid: "4108675c9b884f1cb4f9f052e9bef752" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 33591 } }
I20260809 12:21:48.661393 17044 leader_election.cc:288] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d [CANDIDATE]: Term 1 election: Requested vote from peers eec60939304442ebae620711e9047057 (127.19.183.195:35639), 4108675c9b884f1cb4f9f052e9bef752 (127.19.183.194:33591)
I20260809 12:21:48.661815 16937 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58f9e874159f4cac9a967a487c5d4acc" candidate_uuid: "4a32c96d30a94b358fc2f3708f05bc5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eec60939304442ebae620711e9047057"
I20260809 12:21:48.661907 16747 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "58f9e874159f4cac9a967a487c5d4acc" candidate_uuid: "4a32c96d30a94b358fc2f3708f05bc5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4108675c9b884f1cb4f9f052e9bef752"
I20260809 12:21:48.662024 16937 raft_consensus.cc:3037] T 58f9e874159f4cac9a967a487c5d4acc P eec60939304442ebae620711e9047057 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:48.662132 16747 raft_consensus.cc:3037] T 58f9e874159f4cac9a967a487c5d4acc P 4108675c9b884f1cb4f9f052e9bef752 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:48.664510 16937 raft_consensus.cc:2454] T 58f9e874159f4cac9a967a487c5d4acc P eec60939304442ebae620711e9047057 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a32c96d30a94b358fc2f3708f05bc5d in term 1.
I20260809 12:21:48.664521 16747 raft_consensus.cc:2454] T 58f9e874159f4cac9a967a487c5d4acc P 4108675c9b884f1cb4f9f052e9bef752 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a32c96d30a94b358fc2f3708f05bc5d in term 1.
I20260809 12:21:48.664881 16447 leader_election.cc:302] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a32c96d30a94b358fc2f3708f05bc5d, eec60939304442ebae620711e9047057; no voters:
I20260809 12:21:48.665162 17044 raft_consensus.cc:2781] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:21:48.665585 17044 raft_consensus.cc:686] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d [term 1 LEADER]: Becoming Leader. State: Replica: 4a32c96d30a94b358fc2f3708f05bc5d, State: Running, Role: LEADER
I20260809 12:21:48.666038 17044 consensus_queue.cc:237] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a32c96d30a94b358fc2f3708f05bc5d" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37595 } } peers { permanent_uuid: "eec60939304442ebae620711e9047057" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35639 } } peers { permanent_uuid: "4108675c9b884f1cb4f9f052e9bef752" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 33591 } }
I20260809 12:21:48.670639 16338 catalog_manager.cc:5495] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d reported cstate change: term changed from 0 to 1, leader changed from <none> to 4a32c96d30a94b358fc2f3708f05bc5d (127.19.183.193). New cstate: current_term: 1 leader_uuid: "4a32c96d30a94b358fc2f3708f05bc5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4a32c96d30a94b358fc2f3708f05bc5d" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 37595 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eec60939304442ebae620711e9047057" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4108675c9b884f1cb4f9f052e9bef752" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 33591 } health_report { overall_health: UNKNOWN } } }
I20260809 12:21:48.733829 17031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:48.728489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35292 (local address 127.19.183.193:37595)
0809 12:21:48.728742 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:48.728747 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:48.728767 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:21:48.728907 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:48.728911 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:48.729002 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:48.729169 (+ 167us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:21:48.729526 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.730201 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.731650 (+ 1449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.732142 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.732278 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:48.732769 (+ 491us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:21:48.733259 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:21:48.733416 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:48.733480 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 12:21:48.733563 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260809 12:21:48.741743 16747 raft_consensus.cc:1261] T 58f9e874159f4cac9a967a487c5d4acc P 4108675c9b884f1cb4f9f052e9bef752 [term 1 FOLLOWER]: Refusing update from remote peer 4a32c96d30a94b358fc2f3708f05bc5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:21:48.741818 16937 raft_consensus.cc:1261] T 58f9e874159f4cac9a967a487c5d4acc P eec60939304442ebae620711e9047057 [term 1 FOLLOWER]: Refusing update from remote peer 4a32c96d30a94b358fc2f3708f05bc5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:21:48.742770 17048 consensus_queue.cc:1035] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "4108675c9b884f1cb4f9f052e9bef752" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 33591 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:21:48.743127 17044 consensus_queue.cc:1035] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "eec60939304442ebae620711e9047057" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 35639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:21:48.776445 16337 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:21:48.776818 16337 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46162:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:21:48.780862 16337 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e2e38283e9f24da1ad04524bca12a0a1: Sending DeleteTablet for 3 replicas of tablet 58f9e874159f4cac9a967a487c5d4acc
I20260809 12:21:48.782069 16911 tablet_service.cc:1514] Processing DeleteTablet for tablet 58f9e874159f4cac9a967a487c5d4acc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:21:48 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:33360
I20260809 12:21:48.782027 16512 tablet_service.cc:1514] Processing DeleteTablet for tablet 58f9e874159f4cac9a967a487c5d4acc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:21:48 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:35286
I20260809 12:21:48.782226 16716 tablet_service.cc:1514] Processing DeleteTablet for tablet 58f9e874159f4cac9a967a487c5d4acc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:21:48 UTC) from {username='kudu', principal='kudu/127.19.183.254@KRBTEST.COM'} at 127.0.0.1:44840
I20260809 12:21:48.784111 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 16424
I20260809 12:21:48.792245 17070 tablet_replica.cc:330] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d: stopping tablet replica
I20260809 12:21:48.792754 17070 raft_consensus.cc:2229] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:21:48.793497 17070 raft_consensus.cc:2258] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:21:48.795806 17069 tablet_replica.cc:330] T 58f9e874159f4cac9a967a487c5d4acc P eec60939304442ebae620711e9047057: stopping tablet replica
I20260809 12:21:48.796227 17069 raft_consensus.cc:2229] T 58f9e874159f4cac9a967a487c5d4acc P eec60939304442ebae620711e9047057 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:21:48.796739 17069 raft_consensus.cc:2258] T 58f9e874159f4cac9a967a487c5d4acc P eec60939304442ebae620711e9047057 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:21:48.799131 17069 ts_tablet_manager.cc:1905] T 58f9e874159f4cac9a967a487c5d4acc P eec60939304442ebae620711e9047057: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:21:48.802273 17071 tablet_replica.cc:330] T 58f9e874159f4cac9a967a487c5d4acc P 4108675c9b884f1cb4f9f052e9bef752: stopping tablet replica
I20260809 12:21:48.802629 17070 ts_tablet_manager.cc:1905] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:21:48.802732 17071 raft_consensus.cc:2229] T 58f9e874159f4cac9a967a487c5d4acc P 4108675c9b884f1cb4f9f052e9bef752 [term 1 FOLLOWER]: Raft consensus shutting down.
Aug 09 12:21:48 dist-test-slave-39l8 krb5kdc[16267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.193@KRBTEST.COM
I20260809 12:21:48.803278 17071 raft_consensus.cc:2258] T 58f9e874159f4cac9a967a487c5d4acc P 4108675c9b884f1cb4f9f052e9bef752 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:21:48.808413 17071 ts_tablet_manager.cc:1905] T 58f9e874159f4cac9a967a487c5d4acc P 4108675c9b884f1cb4f9f052e9bef752: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:21:48.811877 17070 ts_tablet_manager.cc:1918] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:21:48.812078 17070 log.cc:1196] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/58f9e874159f4cac9a967a487c5d4acc
I20260809 12:21:48.812827 17070 ts_tablet_manager.cc:1939] T 58f9e874159f4cac9a967a487c5d4acc P 4a32c96d30a94b358fc2f3708f05bc5d: Deleting consensus metadata
I20260809 12:21:48.814824 16316 catalog_manager.cc:4841] TS 4a32c96d30a94b358fc2f3708f05bc5d (127.19.183.193:37595): tablet 58f9e874159f4cac9a967a487c5d4acc (table test-table [id=9209a81e96da411398030da65c8ce388]) successfully deleted
I20260809 12:21:48.815953 17031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:48.786631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35302 (local address 127.19.183.193:37595)
0809 12:21:48.786819 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:48.786825 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:48.791205 (+ 4380us) server_negotiation.cc:407] Connection header received
0809 12:21:48.792019 (+ 814us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:48.792025 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:48.792109 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:48.792217 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:48.794312 (+ 2095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.795802 (+ 1490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.797827 (+ 2025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.798492 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.798595 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:48.805025 (+ 6430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:48.805043 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:48.805060 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:48.805105 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:48.809693 (+ 4588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:48.810861 (+ 1168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:48.810874 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:48.810879 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:48.811073 (+ 194us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:48.811664 (+ 591us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:48.811669 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:48.811672 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:48.812378 (+ 706us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:48.812707 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:48.815279 (+ 2572us) server_negotiation.cc:299] Negotiation successful
0809 12:21:48.815554 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 12:21:48.817601 16604 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:48.818077 17069 ts_tablet_manager.cc:1918] T 58f9e874159f4cac9a967a487c5d4acc P eec60939304442ebae620711e9047057: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:21:48.818271 17069 log.cc:1196] T 58f9e874159f4cac9a967a487c5d4acc P eec60939304442ebae620711e9047057: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/58f9e874159f4cac9a967a487c5d4acc
I20260809 12:21:48.818639 17071 ts_tablet_manager.cc:1918] T 58f9e874159f4cac9a967a487c5d4acc P 4108675c9b884f1cb4f9f052e9bef752: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:21:48.818871 17071 log.cc:1196] T 58f9e874159f4cac9a967a487c5d4acc P 4108675c9b884f1cb4f9f052e9bef752: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/58f9e874159f4cac9a967a487c5d4acc
I20260809 12:21:48.818900 17069 ts_tablet_manager.cc:1939] T 58f9e874159f4cac9a967a487c5d4acc P eec60939304442ebae620711e9047057: Deleting consensus metadata
I20260809 12:21:48.819537 17071 ts_tablet_manager.cc:1939] T 58f9e874159f4cac9a967a487c5d4acc P 4108675c9b884f1cb4f9f052e9bef752: Deleting consensus metadata
I20260809 12:21:48.821305 16319 catalog_manager.cc:4841] TS eec60939304442ebae620711e9047057 (127.19.183.195:35639): tablet 58f9e874159f4cac9a967a487c5d4acc (table test-table [id=9209a81e96da411398030da65c8ce388]) successfully deleted
I20260809 12:21:48.822233 16319 catalog_manager.cc:4841] TS 4108675c9b884f1cb4f9f052e9bef752 (127.19.183.194:33591): tablet 58f9e874159f4cac9a967a487c5d4acc (table test-table [id=9209a81e96da411398030da65c8ce388]) successfully deleted
I20260809 12:21:48.919996 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 16424
I20260809 12:21:48.950520 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 16623
Aug 09 12:21:48 dist-test-slave-39l8 krb5kdc[16267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.194@KRBTEST.COM
I20260809 12:21:48.964569 17034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:48.951337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44842 (local address 127.19.183.194:33591)
0809 12:21:48.951526 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:48.951531 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:48.951549 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:21:48.951714 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:48.951718 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:48.951779 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:48.951863 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:48.953129 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.953835 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.955103 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:48.955489 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:48.955596 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:48.959582 (+ 3986us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:48.959594 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:48.959610 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:48.959655 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:48.962167 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:48.962838 (+ 671us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:48.962846 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:48.962848 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:48.962985 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:48.963313 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:48.963316 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:48.963318 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:48.963683 (+ 365us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:48.963899 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:48.964139 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 12:21:48.964290 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 12:21:48.965423 16807 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:49.055191 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 16623
I20260809 12:21:49.083863 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 16824
Aug 09 12:21:49 dist-test-slave-39l8 krb5kdc[16267](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278105, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.195@KRBTEST.COM
I20260809 12:21:49.098742 17032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:49.084641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33372 (local address 127.19.183.195:35639)
0809 12:21:49.084805 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:49.084809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:49.084829 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:21:49.084975 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:49.084979 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:49.085071 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:49.085159 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:49.086538 (+ 1379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:49.087274 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:49.088502 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:49.088870 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:49.088958 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:49.093557 (+ 4599us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:49.093596 (+ 39us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:49.093614 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:49.093651 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:49.096260 (+ 2609us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:49.096883 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:49.096890 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:49.096893 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:49.097026 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:49.097347 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:49.097350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:49.097352 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:49.097705 (+ 353us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:49.097916 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:49.098186 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 12:21:49.098336 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 12:21:49.099401 17000 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:49.186885 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 16824
I20260809 12:21:49.211768 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 16284
I20260809 12:21:49.212917 16384 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:49.322861 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 16284
2026-08-09T12:21:49Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (4841 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T12:21:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:21:49Z Disabled control of system clock
I20260809 12:21:49.421249 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:40561
--webserver_interface=127.19.183.254
--webserver_port=0
--builtin_ntp_servers=127.19.183.212:46721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:40561
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 12:21:49.756865 17084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:49.757150 17084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:49.757228 17084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:49.763907 17084 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:21:49.763999 17084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:49.764042 17084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:49.764099 17084 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:21:49.764139 17084 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:21:49.772853 17084 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:46721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:40561
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:40561
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:49.773963 17084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:49.775761 17084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:49.787194 17094 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:49.787321 17090 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:49.787236 17091 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:49.788393 17084 server_base.cc:1034] running on GCE node
I20260809 12:21:49.789088 17084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:49.790357 17084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:49.791585 17084 hybrid_clock.cc:648] HybridClock initialized: now 1786278109791546 us; error 46 us; skew 500 ppm
I20260809 12:21:49.791922 17084 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:21:49.794178 17084 webserver.cc:460] Webserver started at http://127.19.183.254:36473/ using document root <none> and password file <none>
I20260809 12:21:49.794952 17084 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:49.795080 17084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:49.795399 17084 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:49.798166 17084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "e674957562864f0e8c93ce07006a4eea"
format_stamp: "Formatted at 2026-08-09 12:21:49 on dist-test-slave-39l8"
server_key: "d6ae0267c9a12af3ff859675f2d37359"
server_key_iv: "1ac58cf65a3c7ec26f6bece118e94665"
server_key_version: "encryptionkey@0"
I20260809 12:21:49.799011 17084 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "e674957562864f0e8c93ce07006a4eea"
format_stamp: "Formatted at 2026-08-09 12:21:49 on dist-test-slave-39l8"
server_key: "d6ae0267c9a12af3ff859675f2d37359"
server_key_iv: "1ac58cf65a3c7ec26f6bece118e94665"
server_key_version: "encryptionkey@0"
I20260809 12:21:49.803973 17084 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260809 12:21:49.807364 17102 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:49.808624 17084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:21:49.808903 17084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "e674957562864f0e8c93ce07006a4eea"
format_stamp: "Formatted at 2026-08-09 12:21:49 on dist-test-slave-39l8"
server_key: "d6ae0267c9a12af3ff859675f2d37359"
server_key_iv: "1ac58cf65a3c7ec26f6bece118e94665"
server_key_version: "encryptionkey@0"
I20260809 12:21:49.809146 17084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:49.826287 17084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:49.827128 17084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:49.827409 17084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:49.846060 17084 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:40561
I20260809 12:21:49.846079 17175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:40561 every 8 connection(s)
I20260809 12:21:49.847442 17084 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:21:49.851222 17176 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:49.856026 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 17084
I20260809 12:21:49.856222 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:21:49.856662 20191 external_mini_cluster.cc:1428] Setting key fc84284de38b00d9d5afbc5fd8f95973
I20260809 12:21:49.863122 17176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea: Bootstrap starting.
I20260809 12:21:49.866852 17176 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:49.868108 17176 log.cc:824] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:49.871645 17176 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea: No bootstrap required, opened a new log
I20260809 12:21:49.872601 17179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:49.858668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42110 (local address 127.19.183.254:40561)
0809 12:21:49.859829 (+ 1161us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:49.859844 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:49.859898 (+ 54us) server_negotiation.cc:407] Connection header received
0809 12:21:49.862341 (+ 2443us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:49.862369 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:49.862938 (+ 569us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:49.863366 (+ 428us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:49.863889 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:49.865870 (+ 1981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:49.867307 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:49.868361 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:49.868564 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:49.869849 (+ 1285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:49.869859 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:49.869875 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:49.870564 (+ 689us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:49.870662 (+ 98us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:49.870692 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:49.870836 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:49.871021 (+ 185us) server_negotiation.cc:299] Negotiation successful
0809 12:21:49.871331 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":351,"threads_started":1}
I20260809 12:21:49.879804 17176 raft_consensus.cc:348] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e674957562864f0e8c93ce07006a4eea" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40561 } }
I20260809 12:21:49.880254 17176 raft_consensus.cc:374] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:49.880398 17176 raft_consensus.cc:729] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e674957562864f0e8c93ce07006a4eea, State: Initialized, Role: FOLLOWER
I20260809 12:21:49.881029 17176 consensus_queue.cc:260] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e674957562864f0e8c93ce07006a4eea" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40561 } }
I20260809 12:21:49.881266 17176 raft_consensus.cc:388] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:21:49.881366 17176 raft_consensus.cc:482] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:21:49.881495 17176 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:49.883900 17176 raft_consensus.cc:504] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e674957562864f0e8c93ce07006a4eea" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40561 } }
I20260809 12:21:49.884419 17176 leader_election.cc:302] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e674957562864f0e8c93ce07006a4eea; no voters:
I20260809 12:21:49.884931 17176 leader_election.cc:288] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:21:49.885085 17181 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:21:49.885811 17181 raft_consensus.cc:686] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea [term 1 LEADER]: Becoming Leader. State: Replica: e674957562864f0e8c93ce07006a4eea, State: Running, Role: LEADER
I20260809 12:21:49.886309 17181 consensus_queue.cc:237] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e674957562864f0e8c93ce07006a4eea" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40561 } }
I20260809 12:21:49.886941 17176 sys_catalog.cc:564] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:21:49.889043 17183 sys_catalog.cc:455] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea [sys.catalog]: SysCatalogTable state changed. Reason: New leader e674957562864f0e8c93ce07006a4eea. Latest consensus state: current_term: 1 leader_uuid: "e674957562864f0e8c93ce07006a4eea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e674957562864f0e8c93ce07006a4eea" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40561 } } }
I20260809 12:21:49.888998 17182 sys_catalog.cc:455] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e674957562864f0e8c93ce07006a4eea" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e674957562864f0e8c93ce07006a4eea" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40561 } } }
I20260809 12:21:49.889585 17182 sys_catalog.cc:458] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea [sys.catalog]: This master's current role is: LEADER
I20260809 12:21:49.891072 17183 sys_catalog.cc:458] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea [sys.catalog]: This master's current role is: LEADER
I20260809 12:21:49.891981 17187 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:21:49.896912 17187 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:21:49.906147 17187 catalog_manager.cc:1347] Generated new cluster ID: 3b3d1af6761c407aa3fbe3fbfe0dd5d6
I20260809 12:21:49.906260 17187 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:21:49.923346 17187 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:21:49.924767 17187 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:21:49.936599 17187 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea: Generated new TSK 0
I20260809 12:21:49.937412 17187 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:21:49.943697 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.193:0
--local_ip_for_outbound_sockets=127.19.183.193
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40561
--builtin_ntp_servers=127.19.183.212:46721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 12:21:50.290477 17204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:50.290817 17204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:50.290968 17204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:50.298070 17204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:50.298182 17204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:50.298382 17204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.193
I20260809 12:21:50.307332 17204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:46721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.193:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.19.183.193
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40561
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.193
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:50.308568 17204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:50.310250 17204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:50.323410 17210 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:50.323453 17209 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:50.323460 17214 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:50.324231 17204 server_base.cc:1034] running on GCE node
I20260809 12:21:50.324882 17204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:50.325795 17204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:50.327036 17204 hybrid_clock.cc:648] HybridClock initialized: now 1786278110326983 us; error 60 us; skew 500 ppm
I20260809 12:21:50.327344 17204 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:21:50.329664 17204 webserver.cc:460] Webserver started at http://127.19.183.193:35035/ using document root <none> and password file <none>
I20260809 12:21:50.330338 17204 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:50.330463 17204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:50.330868 17204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:50.333673 17204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-0/data/instance:
uuid: "30ae1e28f9044a36ade2966831e457f0"
format_stamp: "Formatted at 2026-08-09 12:21:50 on dist-test-slave-39l8"
server_key: "5359ed1355e040fa53354661c0049ff6"
server_key_iv: "26005d70561a1588029c349c540792d5"
server_key_version: "encryptionkey@0"
I20260809 12:21:50.334461 17204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance:
uuid: "30ae1e28f9044a36ade2966831e457f0"
format_stamp: "Formatted at 2026-08-09 12:21:50 on dist-test-slave-39l8"
server_key: "5359ed1355e040fa53354661c0049ff6"
server_key_iv: "26005d70561a1588029c349c540792d5"
server_key_version: "encryptionkey@0"
I20260809 12:21:50.339574 17204 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.003s
I20260809 12:21:50.342947 17222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:50.344151 17204 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:21:50.344313 17204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-0/wal
uuid: "30ae1e28f9044a36ade2966831e457f0"
format_stamp: "Formatted at 2026-08-09 12:21:50 on dist-test-slave-39l8"
server_key: "5359ed1355e040fa53354661c0049ff6"
server_key_iv: "26005d70561a1588029c349c540792d5"
server_key_version: "encryptionkey@0"
I20260809 12:21:50.344542 17204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:50.370636 17204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:50.371417 17204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:50.371722 17204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:50.373113 17204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:21:50.374596 17204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:21:50.374686 17204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:50.374821 17204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:21:50.374882 17204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:50.408138 17204 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.193:32811
I20260809 12:21:50.408222 17374 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.193:32811 every 8 connection(s)
I20260809 12:21:50.409469 17204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-0/data/info.pb
I20260809 12:21:50.417989 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 17204
I20260809 12:21:50.418176 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-0/wal/instance
I20260809 12:21:50.418761 20191 external_mini_cluster.cc:1428] Setting key 7973c7397fca6ad0791f6c4bea2eb5dc
I20260809 12:21:50.421837 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.194:0
--local_ip_for_outbound_sockets=127.19.183.194
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40561
--builtin_ntp_servers=127.19.183.212:46721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 12:21:50.422807 17377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:50.411926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:40193 (local address 127.19.183.254:40561)
0809 12:21:50.412430 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:50.412437 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:50.413747 (+ 1310us) server_negotiation.cc:407] Connection header received
0809 12:21:50.415723 (+ 1976us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:50.415728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:50.415803 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:50.415891 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:50.416881 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:50.417558 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:50.419764 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:50.420331 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:50.420445 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:50.421888 (+ 1443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:50.421893 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:50.421895 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:50.421912 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:50.421951 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:50.421961 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:50.422068 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:50.422334 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 12:21:50.422420 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":227,"threads_started":1}
I20260809 12:21:50.423702 17378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:50.412589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:40561 (local address 127.19.183.193:40193)
0809 12:21:50.413569 (+ 980us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:50.413614 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:50.415453 (+ 1839us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:50.416105 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:50.416112 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:50.416192 (+ 80us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:21:50.416730 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:50.416746 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:50.417729 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:50.417737 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:50.419606 (+ 1869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:50.419622 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:50.420572 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:50.420583 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:50.420835 (+ 252us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:50.421561 (+ 726us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:21:50.421564 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:21:50.421620 (+ 56us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:21:50.421715 (+ 95us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:21:50.422122 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:21:50.422139 (+ 17us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:21:50.422145 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 12:21:50.422318 (+ 173us) client_negotiation.cc:240] Negotiation successful
0809 12:21:50.422731 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":271,"threads_started":1}
I20260809 12:21:50.425107 17375 heartbeater.cc:347] Connected to a master server at 127.19.183.254:40561
I20260809 12:21:50.425377 17375 heartbeater.cc:464] Registering TS with master...
I20260809 12:21:50.426096 17375 heartbeater.cc:511] Master 127.19.183.254:40561 requested a full tablet report, sending...
I20260809 12:21:50.428268 17123 ts_manager.cc:194] Registered new tserver with Master: 30ae1e28f9044a36ade2966831e457f0 (127.19.183.193:32811)
I20260809 12:21:50.430375 17123 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.183.193:40193
W20260809 12:21:50.764357 17379 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:50.764624 17379 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:50.764714 17379 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:50.771770 17379 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:50.771876 17379 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:50.771975 17379 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.194
I20260809 12:21:50.780977 17379 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:46721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.194:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.19.183.194
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40561
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.194
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:50.782233 17379 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:50.784050 17379 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:50.797279 17388 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:50.797242 17389 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:50.797333 17392 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:50.798062 17379 server_base.cc:1034] running on GCE node
I20260809 12:21:50.798661 17379 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:50.799639 17379 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:50.800844 17379 hybrid_clock.cc:648] HybridClock initialized: now 1786278110800801 us; error 41 us; skew 500 ppm
I20260809 12:21:50.801139 17379 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:21:50.803324 17379 webserver.cc:460] Webserver started at http://127.19.183.194:44605/ using document root <none> and password file <none>
I20260809 12:21:50.803958 17379 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:50.804056 17379 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:50.804327 17379 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:50.807279 17379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-1/data/instance:
uuid: "6b4335b687e143bd824a4c2ae02e0f8f"
format_stamp: "Formatted at 2026-08-09 12:21:50 on dist-test-slave-39l8"
server_key: "151137cc731d09c3f22d49a9b77f2bf8"
server_key_iv: "35f9d19a4f5bd64846fb07bc334c4401"
server_key_version: "encryptionkey@0"
I20260809 12:21:50.808038 17379 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance:
uuid: "6b4335b687e143bd824a4c2ae02e0f8f"
format_stamp: "Formatted at 2026-08-09 12:21:50 on dist-test-slave-39l8"
server_key: "151137cc731d09c3f22d49a9b77f2bf8"
server_key_iv: "35f9d19a4f5bd64846fb07bc334c4401"
server_key_version: "encryptionkey@0"
I20260809 12:21:50.813174 17379 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:21:50.816843 17400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:50.818483 17379 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:21:50.818673 17379 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-1/wal
uuid: "6b4335b687e143bd824a4c2ae02e0f8f"
format_stamp: "Formatted at 2026-08-09 12:21:50 on dist-test-slave-39l8"
server_key: "151137cc731d09c3f22d49a9b77f2bf8"
server_key_iv: "35f9d19a4f5bd64846fb07bc334c4401"
server_key_version: "encryptionkey@0"
I20260809 12:21:50.818949 17379 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:50.851279 17379 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:50.852154 17379 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:50.852466 17379 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:50.853472 17379 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:21:50.855077 17379 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:21:50.855204 17379 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:50.855336 17379 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:21:50.855422 17379 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:50.889182 17379 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.194:41163
I20260809 12:21:50.889267 17554 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.194:41163 every 8 connection(s)
I20260809 12:21:50.890501 17379 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-1/data/info.pb
I20260809 12:21:50.896485 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 17379
I20260809 12:21:50.896672 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-1/wal/instance
I20260809 12:21:50.897292 20191 external_mini_cluster.cc:1428] Setting key 3f3b1de6593723e9d80763839d5501d2
I20260809 12:21:50.900230 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.19.183.195:0
--local_ip_for_outbound_sockets=127.19.183.195
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40561
--builtin_ntp_servers=127.19.183.212:46721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 12:21:50.903915 17377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:50.893168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.194:44213 (local address 127.19.183.254:40561)
0809 12:21:50.893344 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:50.893350 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:50.894761 (+ 1411us) server_negotiation.cc:407] Connection header received
0809 12:21:50.896623 (+ 1862us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:50.896629 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:50.896716 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:50.896832 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:50.898260 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:50.899173 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:50.900861 (+ 1688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:50.901582 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:50.901742 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:50.903014 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:50.903022 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:50.903029 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:50.903053 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:50.903092 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:50.903109 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:50.903192 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:50.903397 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 12:21:50.903520 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 12:21:50.904696 17561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:50.893686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:40561 (local address 127.19.183.194:44213)
0809 12:21:50.894573 (+ 887us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:50.894625 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:50.896327 (+ 1702us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:50.897128 (+ 801us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:50.897139 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:50.897247 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:21:50.898047 (+ 800us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:50.898068 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:50.899329 (+ 1261us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:50.899336 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:50.900707 (+ 1371us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:50.900725 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:50.901798 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:50.901803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:50.902000 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:50.902669 (+ 669us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:21:50.902672 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:21:50.902745 (+ 73us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:21:50.902825 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:21:50.903208 (+ 383us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:21:50.903217 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:21:50.903224 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 12:21:50.903355 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 12:21:50.903644 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":449,"thread_start_us":254,"threads_started":1}
I20260809 12:21:50.906037 17556 heartbeater.cc:347] Connected to a master server at 127.19.183.254:40561
I20260809 12:21:50.906324 17556 heartbeater.cc:464] Registering TS with master...
I20260809 12:21:50.907059 17556 heartbeater.cc:511] Master 127.19.183.254:40561 requested a full tablet report, sending...
I20260809 12:21:50.908365 17123 ts_manager.cc:194] Registered new tserver with Master: 6b4335b687e143bd824a4c2ae02e0f8f (127.19.183.194:41163)
I20260809 12:21:50.909145 17123 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.183.194:44213
W20260809 12:21:51.242365 17562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:51.242699 17562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:51.242888 17562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:51.250192 17562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:51.250317 17562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:51.250484 17562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.19.183.195
I20260809 12:21:51.259528 17562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:46721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.19.183.195:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.19.183.195
--webserver_port=0
--tserver_master_addrs=127.19.183.254:40561
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.19.183.195
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:51.260840 17562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:51.262624 17562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:51.278884 17570 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:51.274859 17569 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:51.280261 17573 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:51.280699 17562 server_base.cc:1034] running on GCE node
I20260809 12:21:51.281291 17562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:51.282181 17562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:51.283406 17562 hybrid_clock.cc:648] HybridClock initialized: now 1786278111283367 us; error 45 us; skew 500 ppm
I20260809 12:21:51.283768 17562 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
I20260809 12:21:51.285867 17562 webserver.cc:460] Webserver started at http://127.19.183.195:43589/ using document root <none> and password file <none>
I20260809 12:21:51.286482 17562 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:51.286616 17562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:51.287022 17562 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:51.290019 17562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-2/data/instance:
uuid: "09b46afc0f4349c7b3ce9741ece0b6b6"
format_stamp: "Formatted at 2026-08-09 12:21:51 on dist-test-slave-39l8"
server_key: "f288f758dfc5dee6087e9cc6437e5db1"
server_key_iv: "d931c0c0da842bb2c14ac24d62500cc4"
server_key_version: "encryptionkey@0"
I20260809 12:21:51.290850 17562 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance:
uuid: "09b46afc0f4349c7b3ce9741ece0b6b6"
format_stamp: "Formatted at 2026-08-09 12:21:51 on dist-test-slave-39l8"
server_key: "f288f758dfc5dee6087e9cc6437e5db1"
server_key_iv: "d931c0c0da842bb2c14ac24d62500cc4"
server_key_version: "encryptionkey@0"
I20260809 12:21:51.295982 17562 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:21:51.299536 17581 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:51.300841 17562 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:21:51.300972 17562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-2/wal
uuid: "09b46afc0f4349c7b3ce9741ece0b6b6"
format_stamp: "Formatted at 2026-08-09 12:21:51 on dist-test-slave-39l8"
server_key: "f288f758dfc5dee6087e9cc6437e5db1"
server_key_iv: "d931c0c0da842bb2c14ac24d62500cc4"
server_key_version: "encryptionkey@0"
I20260809 12:21:51.301148 17562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:51.322036 17562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:51.322968 17562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:51.323272 17562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:51.324358 17562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 12:21:51.325858 17562 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 12:21:51.325973 17562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:51.326095 17562 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 12:21:51.326195 17562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:51.359505 17562 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.195:42233
I20260809 12:21:51.359597 17749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.195:42233 every 8 connection(s)
I20260809 12:21:51.360783 17562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-2/data/info.pb
I20260809 12:21:51.365260 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 17562
I20260809 12:21:51.365460 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-2/wal/instance
I20260809 12:21:51.366060 20191 external_mini_cluster.cc:1428] Setting key d8a2dd72f5eff4cc2254b6ec6954779b
I20260809 12:21:51.373695 17377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:51.363503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:47509 (local address 127.19.183.254:40561)
0809 12:21:51.363691 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:51.363697 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:51.365117 (+ 1420us) server_negotiation.cc:407] Connection header received
0809 12:21:51.367585 (+ 2468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:51.367588 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:51.367682 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:51.367796 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:51.369011 (+ 1215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.369726 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.371283 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.371706 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.371797 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:51.373027 (+ 1230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:51.373031 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:51.373035 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:51.373053 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:51.373083 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:51.373094 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:51.373167 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:51.373366 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 12:21:51.373438 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:21:51.374645 17753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:51.363978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:40561 (local address 127.19.183.195:47509)
0809 12:21:51.364922 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:51.364989 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:51.367284 (+ 2295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:51.368054 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:51.368069 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:51.368164 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:21:51.368846 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:51.368872 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.369844 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.369847 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:51.371168 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:51.371180 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.371880 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.371884 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:51.372078 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:51.372713 (+ 635us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 12:21:51.372715 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:21:51.372764 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 12:21:51.372844 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:21:51.373180 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:21:51.373188 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:21:51.373194 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 12:21:51.373340 (+ 146us) client_negotiation.cc:240] Negotiation successful
0809 12:21:51.373605 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":462,"thread_start_us":273,"threads_started":1}
I20260809 12:21:51.376078 17751 heartbeater.cc:347] Connected to a master server at 127.19.183.254:40561
I20260809 12:21:51.376400 17751 heartbeater.cc:464] Registering TS with master...
I20260809 12:21:51.377138 17751 heartbeater.cc:511] Master 127.19.183.254:40561 requested a full tablet report, sending...
I20260809 12:21:51.378448 17123 ts_manager.cc:194] Registered new tserver with Master: 09b46afc0f4349c7b3ce9741ece0b6b6 (127.19.183.195:42233)
I20260809 12:21:51.379371 17123 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.19.183.195:47509
I20260809 12:21:51.381156 20191 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 12:21:51.390461 17377 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:51.385036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47188 (local address 127.19.183.254:40561)
0809 12:21:51.385238 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:51.385247 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:51.385665 (+ 418us) server_negotiation.cc:407] Connection header received
0809 12:21:51.385802 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:51.385805 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:51.385873 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:51.385946 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:51.386398 (+ 452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.387130 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.388497 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.388885 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.388973 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:51.389861 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:51.389865 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:51.389868 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:51.389883 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:51.389910 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:51.389920 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:51.389983 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:51.390089 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 12:21:51.390155 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 12:21:51.393872 17121 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47188:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 12:21:51.400645 17121 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 12:21:51.432451 17769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:51.412933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46270 (local address 127.19.183.193:32811)
0809 12:21:51.424966 (+ 12033us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:51.424971 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:51.424997 (+ 26us) server_negotiation.cc:407] Connection header received
0809 12:21:51.425077 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:51.425082 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:51.425249 (+ 167us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:51.425473 (+ 224us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:51.426178 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.427596 (+ 1418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.429608 (+ 2012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.431150 (+ 1542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.431344 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:51.431536 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:51.431866 (+ 330us) server_negotiation.cc:299] Negotiation successful
0809 12:21:51.432073 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11884,"thread_start_us":11755,"threads_started":1}
I20260809 12:21:51.432487 17770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:51.412671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:32811 (local address 127.0.0.1:46270)
0809 12:21:51.414059 (+ 1388us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:51.414096 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:51.414239 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:51.425470 (+ 11231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:51.425477 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:51.425539 (+ 62us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:51.425926 (+ 387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:51.425939 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.427729 (+ 1790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.427737 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:51.429423 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:51.429437 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.431354 (+ 1917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.431366 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:51.431584 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:51.431642 (+ 58us) client_negotiation.cc:769] Sending connection context
0809 12:21:51.431803 (+ 161us) client_negotiation.cc:240] Negotiation successful
0809 12:21:51.432016 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1094,"mutex_wait_us":1,"spinlock_wait_cycles":17792,"thread_start_us":13951,"threads_started":1}
I20260809 12:21:51.433228 17375 heartbeater.cc:503] Master 127.19.183.254:40561 was elected leader, sending a full tablet report...
I20260809 12:21:51.436172 17767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:51.415004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:42233 (local address 127.0.0.1:33780)
0809 12:21:51.426915 (+ 11911us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:51.426932 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:51.427048 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:51.428678 (+ 1630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:51.428682 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:51.428708 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:51.429137 (+ 429us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:51.429147 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.430866 (+ 1719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.430872 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:51.432961 (+ 2089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:51.433017 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.435308 (+ 2291us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.435315 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:51.435503 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:51.435546 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 12:21:51.435668 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 12:21:51.435788 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11785,"mutex_wait_us":52,"spinlock_wait_cycles":1920,"thread_start_us":10663,"threads_started":1}
I20260809 12:21:51.436277 17775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:51.415354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33780 (local address 127.19.183.195:42233)
0809 12:21:51.428252 (+ 12898us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:51.428258 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:51.428285 (+ 27us) server_negotiation.cc:407] Connection header received
0809 12:21:51.428355 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:51.428360 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:51.428502 (+ 142us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:51.428687 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:51.429362 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.430675 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.433223 (+ 2548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.434651 (+ 1428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.434957 (+ 306us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:51.435172 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:51.435718 (+ 546us) server_negotiation.cc:299] Negotiation successful
0809 12:21:51.435884 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":12724,"thread_start_us":12539,"threads_started":1}
I20260809 12:21:51.436581 17286 tablet_service.cc:1467] Processing CreateTablet for tablet 1abcd7b4961c44aa80690298f27b0ea3 (DEFAULT_TABLE table=test-table [id=659d5933a74e497483ccf6849d43ffee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:21:51.438212 17774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:51.413260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:41163 (local address 127.0.0.1:50976)
0809 12:21:51.425332 (+ 12072us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:51.425364 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:51.425541 (+ 177us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:51.426961 (+ 1420us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:51.426969 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:51.427034 (+ 65us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:51.427553 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:51.427568 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.431644 (+ 4076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.431656 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:51.435552 (+ 3896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:51.435568 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.437428 (+ 1860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.437437 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:51.437617 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:51.437649 (+ 32us) client_negotiation.cc:769] Sending connection context
0809 12:21:51.437737 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 12:21:51.437863 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11872,"mutex_wait_us":38,"spinlock_wait_cycles":12800,"thread_start_us":363,"threads_started":1}
I20260809 12:21:51.438367 17772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:51.413622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50976 (local address 127.19.183.194:41163)
0809 12:21:51.423108 (+ 9486us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:51.423119 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:51.425722 (+ 2603us) server_negotiation.cc:407] Connection header received
0809 12:21:51.425855 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:51.425866 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:51.426177 (+ 311us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:51.426392 (+ 215us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:51.429673 (+ 3281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.431526 (+ 1853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.435805 (+ 4279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.437185 (+ 1380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.437379 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:51.437581 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:51.437822 (+ 241us) server_negotiation.cc:299] Negotiation successful
0809 12:21:51.438003 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9319,"thread_start_us":9228,"threads_started":1}
I20260809 12:21:51.438665 17286 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1abcd7b4961c44aa80690298f27b0ea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:51.440025 17657 tablet_service.cc:1467] Processing CreateTablet for tablet 1abcd7b4961c44aa80690298f27b0ea3 (DEFAULT_TABLE table=test-table [id=659d5933a74e497483ccf6849d43ffee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:21:51.441265 17657 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1abcd7b4961c44aa80690298f27b0ea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:51.442035 17471 tablet_service.cc:1467] Processing CreateTablet for tablet 1abcd7b4961c44aa80690298f27b0ea3 (DEFAULT_TABLE table=test-table [id=659d5933a74e497483ccf6849d43ffee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 12:21:51.443394 17471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1abcd7b4961c44aa80690298f27b0ea3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:51.451853 17778 tablet_bootstrap.cc:492] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6: Bootstrap starting.
I20260809 12:21:51.454914 17778 tablet_bootstrap.cc:654] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:51.455467 17779 tablet_bootstrap.cc:492] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0: Bootstrap starting.
I20260809 12:21:51.457171 17778 log.cc:824] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:51.457731 17780 tablet_bootstrap.cc:492] T 1abcd7b4961c44aa80690298f27b0ea3 P 6b4335b687e143bd824a4c2ae02e0f8f: Bootstrap starting.
I20260809 12:21:51.459316 17779 tablet_bootstrap.cc:654] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:51.460537 17779 log.cc:824] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:51.460798 17780 tablet_bootstrap.cc:654] T 1abcd7b4961c44aa80690298f27b0ea3 P 6b4335b687e143bd824a4c2ae02e0f8f: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:51.461468 17778 tablet_bootstrap.cc:492] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6: No bootstrap required, opened a new log
I20260809 12:21:51.461723 17778 ts_tablet_manager.cc:1397] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6: Time spent bootstrapping tablet: real 0.010s user 0.005s sys 0.002s
I20260809 12:21:51.461972 17780 log.cc:824] T 1abcd7b4961c44aa80690298f27b0ea3 P 6b4335b687e143bd824a4c2ae02e0f8f: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:51.463582 17779 tablet_bootstrap.cc:492] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0: No bootstrap required, opened a new log
I20260809 12:21:51.463827 17779 ts_tablet_manager.cc:1397] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0: Time spent bootstrapping tablet: real 0.009s user 0.006s sys 0.001s
I20260809 12:21:51.464691 17780 tablet_bootstrap.cc:492] T 1abcd7b4961c44aa80690298f27b0ea3 P 6b4335b687e143bd824a4c2ae02e0f8f: No bootstrap required, opened a new log
I20260809 12:21:51.464962 17780 ts_tablet_manager.cc:1397] T 1abcd7b4961c44aa80690298f27b0ea3 P 6b4335b687e143bd824a4c2ae02e0f8f: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260809 12:21:51.471715 17780 raft_consensus.cc:348] T 1abcd7b4961c44aa80690298f27b0ea3 P 6b4335b687e143bd824a4c2ae02e0f8f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae1e28f9044a36ade2966831e457f0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 32811 } } peers { permanent_uuid: "6b4335b687e143bd824a4c2ae02e0f8f" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41163 } } peers { permanent_uuid: "09b46afc0f4349c7b3ce9741ece0b6b6" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 42233 } }
I20260809 12:21:51.472084 17780 raft_consensus.cc:374] T 1abcd7b4961c44aa80690298f27b0ea3 P 6b4335b687e143bd824a4c2ae02e0f8f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:51.472209 17780 raft_consensus.cc:729] T 1abcd7b4961c44aa80690298f27b0ea3 P 6b4335b687e143bd824a4c2ae02e0f8f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6b4335b687e143bd824a4c2ae02e0f8f, State: Initialized, Role: FOLLOWER
I20260809 12:21:51.472635 17778 raft_consensus.cc:348] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae1e28f9044a36ade2966831e457f0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 32811 } } peers { permanent_uuid: "6b4335b687e143bd824a4c2ae02e0f8f" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41163 } } peers { permanent_uuid: "09b46afc0f4349c7b3ce9741ece0b6b6" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 42233 } }
I20260809 12:21:51.472798 17780 consensus_queue.cc:260] T 1abcd7b4961c44aa80690298f27b0ea3 P 6b4335b687e143bd824a4c2ae02e0f8f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae1e28f9044a36ade2966831e457f0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 32811 } } peers { permanent_uuid: "6b4335b687e143bd824a4c2ae02e0f8f" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41163 } } peers { permanent_uuid: "09b46afc0f4349c7b3ce9741ece0b6b6" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 42233 } }
I20260809 12:21:51.473065 17778 raft_consensus.cc:374] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:51.473188 17778 raft_consensus.cc:729] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09b46afc0f4349c7b3ce9741ece0b6b6, State: Initialized, Role: FOLLOWER
I20260809 12:21:51.473886 17778 consensus_queue.cc:260] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae1e28f9044a36ade2966831e457f0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 32811 } } peers { permanent_uuid: "6b4335b687e143bd824a4c2ae02e0f8f" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41163 } } peers { permanent_uuid: "09b46afc0f4349c7b3ce9741ece0b6b6" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 42233 } }
I20260809 12:21:51.473862 17779 raft_consensus.cc:348] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae1e28f9044a36ade2966831e457f0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 32811 } } peers { permanent_uuid: "6b4335b687e143bd824a4c2ae02e0f8f" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41163 } } peers { permanent_uuid: "09b46afc0f4349c7b3ce9741ece0b6b6" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 42233 } }
I20260809 12:21:51.474156 17556 heartbeater.cc:503] Master 127.19.183.254:40561 was elected leader, sending a full tablet report...
I20260809 12:21:51.474291 17779 raft_consensus.cc:374] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:51.474390 17779 raft_consensus.cc:729] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 30ae1e28f9044a36ade2966831e457f0, State: Initialized, Role: FOLLOWER
I20260809 12:21:51.474880 17780 ts_tablet_manager.cc:1428] T 1abcd7b4961c44aa80690298f27b0ea3 P 6b4335b687e143bd824a4c2ae02e0f8f: Time spent starting tablet: real 0.010s user 0.010s sys 0.000s
I20260809 12:21:51.475075 17779 consensus_queue.cc:260] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae1e28f9044a36ade2966831e457f0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 32811 } } peers { permanent_uuid: "6b4335b687e143bd824a4c2ae02e0f8f" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41163 } } peers { permanent_uuid: "09b46afc0f4349c7b3ce9741ece0b6b6" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 42233 } }
I20260809 12:21:51.476783 17751 heartbeater.cc:503] Master 127.19.183.254:40561 was elected leader, sending a full tablet report...
I20260809 12:21:51.477623 17778 ts_tablet_manager.cc:1428] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6: Time spent starting tablet: real 0.016s user 0.013s sys 0.003s
I20260809 12:21:51.479190 17779 ts_tablet_manager.cc:1428] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0: Time spent starting tablet: real 0.015s user 0.008s sys 0.008s
I20260809 12:21:51.480824 17789 raft_consensus.cc:482] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:21:51.481207 17787 raft_consensus.cc:482] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 12:21:51.481161 17789 raft_consensus.cc:504] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae1e28f9044a36ade2966831e457f0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 32811 } } peers { permanent_uuid: "6b4335b687e143bd824a4c2ae02e0f8f" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41163 } } peers { permanent_uuid: "09b46afc0f4349c7b3ce9741ece0b6b6" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 42233 } }
I20260809 12:21:51.481519 17787 raft_consensus.cc:504] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae1e28f9044a36ade2966831e457f0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 32811 } } peers { permanent_uuid: "6b4335b687e143bd824a4c2ae02e0f8f" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41163 } } peers { permanent_uuid: "09b46afc0f4349c7b3ce9741ece0b6b6" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 42233 } }
I20260809 12:21:51.483089 17789 leader_election.cc:288] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6b4335b687e143bd824a4c2ae02e0f8f (127.19.183.194:41163), 09b46afc0f4349c7b3ce9741ece0b6b6 (127.19.183.195:42233)
I20260809 12:21:51.486838 17787 leader_election.cc:288] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 30ae1e28f9044a36ade2966831e457f0 (127.19.183.193:32811), 6b4335b687e143bd824a4c2ae02e0f8f (127.19.183.194:41163)
I20260809 12:21:51.494962 17775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:51.483656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:34401 (local address 127.19.183.195:42233)
0809 12:21:51.484183 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:51.484189 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:51.487945 (+ 3756us) server_negotiation.cc:407] Connection header received
0809 12:21:51.488265 (+ 320us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:51.488274 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:51.488338 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:51.488402 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:51.489223 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.490309 (+ 1086us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.492624 (+ 2315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.493956 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.494117 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:51.494187 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:51.494514 (+ 327us) server_negotiation.cc:299] Negotiation successful
0809 12:21:51.494671 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":9,"server-negotiator.queue_time_us":413}
I20260809 12:21:51.495049 17791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:51.483292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.195:42233 (local address 127.19.183.193:34401)
0809 12:21:51.487859 (+ 4567us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:51.487877 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:51.488101 (+ 224us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:51.488599 (+ 498us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:51.488605 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:51.488636 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:51.489047 (+ 411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:51.489061 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.490481 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.490488 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:51.492432 (+ 1944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:51.492444 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.494132 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.494137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:51.494326 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:51.494363 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 12:21:51.494467 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 12:21:51.494639 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4442,"thread_start_us":4278,"threads_started":1}
I20260809 12:21:51.496201 17753 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:51.483562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.193:32811 (local address 127.19.183.195:33531)
0809 12:21:51.483813 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:51.483833 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:51.483938 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:51.488806 (+ 4868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:51.488811 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:51.488834 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:51.489057 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:51.489069 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.490987 (+ 1918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.490996 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:51.493694 (+ 2698us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:51.493716 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.495391 (+ 1675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.495399 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:51.495615 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:51.495654 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 12:21:51.495760 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 12:21:51.495901 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"mutex_wait_us":10}
I20260809 12:21:51.496111 17687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1abcd7b4961c44aa80690298f27b0ea3" candidate_uuid: "30ae1e28f9044a36ade2966831e457f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b46afc0f4349c7b3ce9741ece0b6b6" is_pre_election: true
I20260809 12:21:51.496488 17769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:51.488140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:33531 (local address 127.19.183.193:32811)
0809 12:21:51.488445 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:51.488450 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:51.488467 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:21:51.488521 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:51.488525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:51.488629 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:51.488753 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:51.489246 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.490778 (+ 1532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.493915 (+ 3137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.495050 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.495218 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:51.495295 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:51.496138 (+ 843us) server_negotiation.cc:299] Negotiation successful
0809 12:21:51.496274 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187}
I20260809 12:21:51.496630 17687 raft_consensus.cc:2454] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 30ae1e28f9044a36ade2966831e457f0 in term 0.
I20260809 12:21:51.497046 17794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:51.484226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:41163 (local address 127.19.183.195:55617)
0809 12:21:51.484784 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:51.484804 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:51.484931 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:51.488059 (+ 3128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:51.488063 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:51.488088 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:51.488537 (+ 449us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:51.488550 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.490404 (+ 1854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.490411 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:51.493517 (+ 3106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:51.493532 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.495956 (+ 2424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.495968 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:51.496127 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:51.496156 (+ 29us) client_negotiation.cc:769] Sending connection context
0809 12:21:51.496257 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 12:21:51.496407 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"mutex_wait_us":832,"thread_start_us":542,"threads_started":1}
I20260809 12:21:51.497329 17796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:51.484598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.195:55617 (local address 127.19.183.194:41163)
0809 12:21:51.487682 (+ 3084us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:51.487689 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:51.487714 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:21:51.487781 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:51.487785 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:51.487852 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:51.488698 (+ 846us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:51.488973 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.490218 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.494270 (+ 4052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.495780 (+ 1510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.495969 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:51.496039 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:51.496850 (+ 811us) server_negotiation.cc:299] Negotiation successful
0809 12:21:51.496986 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2956,"thread_start_us":2725,"threads_started":1}
I20260809 12:21:51.497377 17314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1abcd7b4961c44aa80690298f27b0ea3" candidate_uuid: "09b46afc0f4349c7b3ce9741ece0b6b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ae1e28f9044a36ade2966831e457f0" is_pre_election: true
I20260809 12:21:51.497759 17314 raft_consensus.cc:2454] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09b46afc0f4349c7b3ce9741ece0b6b6 in term 0.
I20260809 12:21:51.498338 17587 leader_election.cc:302] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09b46afc0f4349c7b3ce9741ece0b6b6, 30ae1e28f9044a36ade2966831e457f0; no voters:
I20260809 12:21:51.498338 17223 leader_election.cc:302] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09b46afc0f4349c7b3ce9741ece0b6b6, 30ae1e28f9044a36ade2966831e457f0; no voters:
I20260809 12:21:51.498323 17495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1abcd7b4961c44aa80690298f27b0ea3" candidate_uuid: "09b46afc0f4349c7b3ce9741ece0b6b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b4335b687e143bd824a4c2ae02e0f8f" is_pre_election: true
I20260809 12:21:51.498725 17789 raft_consensus.cc:2781] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:21:51.498883 17789 raft_consensus.cc:482] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:21:51.498970 17789 raft_consensus.cc:3037] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:51.498967 17495 raft_consensus.cc:2454] T 1abcd7b4961c44aa80690298f27b0ea3 P 6b4335b687e143bd824a4c2ae02e0f8f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09b46afc0f4349c7b3ce9741ece0b6b6 in term 0.
I20260809 12:21:51.499414 17787 raft_consensus.cc:2781] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 12:21:51.499567 17787 raft_consensus.cc:482] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 12:21:51.499652 17787 raft_consensus.cc:3037] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:51.501322 17789 raft_consensus.cc:504] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae1e28f9044a36ade2966831e457f0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 32811 } } peers { permanent_uuid: "6b4335b687e143bd824a4c2ae02e0f8f" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41163 } } peers { permanent_uuid: "09b46afc0f4349c7b3ce9741ece0b6b6" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 42233 } }
I20260809 12:21:51.501894 17789 leader_election.cc:288] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0 [CANDIDATE]: Term 1 election: Requested vote from peers 6b4335b687e143bd824a4c2ae02e0f8f (127.19.183.194:41163), 09b46afc0f4349c7b3ce9741ece0b6b6 (127.19.183.195:42233)
I20260809 12:21:51.502595 17687 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1abcd7b4961c44aa80690298f27b0ea3" candidate_uuid: "30ae1e28f9044a36ade2966831e457f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09b46afc0f4349c7b3ce9741ece0b6b6"
I20260809 12:21:51.502743 17787 raft_consensus.cc:504] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae1e28f9044a36ade2966831e457f0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 32811 } } peers { permanent_uuid: "6b4335b687e143bd824a4c2ae02e0f8f" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41163 } } peers { permanent_uuid: "09b46afc0f4349c7b3ce9741ece0b6b6" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 42233 } }
I20260809 12:21:51.503135 17687 raft_consensus.cc:2379] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 30ae1e28f9044a36ade2966831e457f0 in current term 1: Already voted for candidate 09b46afc0f4349c7b3ce9741ece0b6b6 in this term.
I20260809 12:21:51.503777 17787 leader_election.cc:288] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 [CANDIDATE]: Term 1 election: Requested vote from peers 30ae1e28f9044a36ade2966831e457f0 (127.19.183.193:32811), 6b4335b687e143bd824a4c2ae02e0f8f (127.19.183.194:41163)
I20260809 12:21:51.503759 17314 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1abcd7b4961c44aa80690298f27b0ea3" candidate_uuid: "09b46afc0f4349c7b3ce9741ece0b6b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "30ae1e28f9044a36ade2966831e457f0"
I20260809 12:21:51.503866 17495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1abcd7b4961c44aa80690298f27b0ea3" candidate_uuid: "09b46afc0f4349c7b3ce9741ece0b6b6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b4335b687e143bd824a4c2ae02e0f8f"
I20260809 12:21:51.503998 17314 raft_consensus.cc:2379] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 09b46afc0f4349c7b3ce9741ece0b6b6 in current term 1: Already voted for candidate 30ae1e28f9044a36ade2966831e457f0 in this term.
I20260809 12:21:51.504053 17495 raft_consensus.cc:3037] T 1abcd7b4961c44aa80690298f27b0ea3 P 6b4335b687e143bd824a4c2ae02e0f8f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:51.504855 17772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:51.484033 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.19.183.193:53141 (local address 127.19.183.194:41163)
0809 12:21:51.484252 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:51.484261 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:51.495537 (+ 11276us) server_negotiation.cc:407] Connection header received
0809 12:21:51.497429 (+ 1892us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:51.497434 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:51.497549 (+ 115us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:51.497635 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 12:21:51.498710 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.499709 (+ 999us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.502263 (+ 2554us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.503309 (+ 1046us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.503454 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:51.503516 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:51.504425 (+ 909us) server_negotiation.cc:299] Negotiation successful
0809 12:21:51.504576 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 12:21:51.505378 17793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:51.483445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.194:41163 (local address 127.19.183.193:53141)
0809 12:21:51.495445 (+ 12000us) negotiation.cc:107] Waiting for socket to connect
0809 12:21:51.495462 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 12:21:51.497304 (+ 1842us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:21:51.497963 (+ 659us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:21:51.497968 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:21:51.498006 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 12:21:51.498446 (+ 440us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:51.498470 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.499923 (+ 1453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.499933 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:51.502073 (+ 2140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:21:51.502088 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.503667 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.503674 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:21:51.504209 (+ 535us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:51.504256 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 12:21:51.505072 (+ 816us) client_negotiation.cc:240] Negotiation successful
0809 12:21:51.505164 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11867,"thread_start_us":10364,"threads_started":1}
I20260809 12:21:51.506155 17494 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1abcd7b4961c44aa80690298f27b0ea3" candidate_uuid: "30ae1e28f9044a36ade2966831e457f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b4335b687e143bd824a4c2ae02e0f8f" is_pre_election: true
I20260809 12:21:51.506369 17493 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1abcd7b4961c44aa80690298f27b0ea3" candidate_uuid: "30ae1e28f9044a36ade2966831e457f0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6b4335b687e143bd824a4c2ae02e0f8f"
I20260809 12:21:51.507051 17495 raft_consensus.cc:2454] T 1abcd7b4961c44aa80690298f27b0ea3 P 6b4335b687e143bd824a4c2ae02e0f8f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09b46afc0f4349c7b3ce9741ece0b6b6 in term 1.
I20260809 12:21:51.507426 17583 leader_election.cc:302] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 09b46afc0f4349c7b3ce9741ece0b6b6, 6b4335b687e143bd824a4c2ae02e0f8f; no voters: 30ae1e28f9044a36ade2966831e457f0
I20260809 12:21:51.507493 17224 leader_election.cc:302] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 30ae1e28f9044a36ade2966831e457f0; no voters: 09b46afc0f4349c7b3ce9741ece0b6b6, 6b4335b687e143bd824a4c2ae02e0f8f
I20260809 12:21:51.507750 17789 raft_consensus.cc:2726] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 12:21:51.508057 17787 raft_consensus.cc:2781] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:21:51.508601 17787 raft_consensus.cc:686] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 [term 1 LEADER]: Becoming Leader. State: Replica: 09b46afc0f4349c7b3ce9741ece0b6b6, State: Running, Role: LEADER
I20260809 12:21:51.509055 17787 consensus_queue.cc:237] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae1e28f9044a36ade2966831e457f0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 32811 } } peers { permanent_uuid: "6b4335b687e143bd824a4c2ae02e0f8f" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41163 } } peers { permanent_uuid: "09b46afc0f4349c7b3ce9741ece0b6b6" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 42233 } }
I20260809 12:21:51.513621 17123 catalog_manager.cc:5495] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 09b46afc0f4349c7b3ce9741ece0b6b6 (127.19.183.195). New cstate: current_term: 1 leader_uuid: "09b46afc0f4349c7b3ce9741ece0b6b6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "30ae1e28f9044a36ade2966831e457f0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 32811 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6b4335b687e143bd824a4c2ae02e0f8f" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41163 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09b46afc0f4349c7b3ce9741ece0b6b6" member_type: VOTER last_known_addr { host: "127.19.183.195" port: 42233 } health_report { overall_health: HEALTHY } } }
I20260809 12:21:51.534540 17775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:51.529587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33788 (local address 127.19.183.195:42233)
0809 12:21:51.529754 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:51.529758 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:51.529774 (+ 16us) server_negotiation.cc:407] Connection header received
0809 12:21:51.529942 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:51.529945 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:51.530077 (+ 132us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:51.530202 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:21:51.530514 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.531348 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.532591 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.533009 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.533107 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:51.533512 (+ 405us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:21:51.533954 (+ 442us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:21:51.534079 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:51.534179 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 12:21:51.534234 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 12:21:51.541836 17495 raft_consensus.cc:1261] T 1abcd7b4961c44aa80690298f27b0ea3 P 6b4335b687e143bd824a4c2ae02e0f8f [term 1 FOLLOWER]: Refusing update from remote peer 09b46afc0f4349c7b3ce9741ece0b6b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:21:51.541853 17314 raft_consensus.cc:1261] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0 [term 1 FOLLOWER]: Refusing update from remote peer 09b46afc0f4349c7b3ce9741ece0b6b6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:21:51.542582 17797 consensus_queue.cc:1035] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "30ae1e28f9044a36ade2966831e457f0" member_type: VOTER last_known_addr { host: "127.19.183.193" port: 32811 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:21:51.542956 17787 consensus_queue.cc:1035] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6b4335b687e143bd824a4c2ae02e0f8f" member_type: VOTER last_known_addr { host: "127.19.183.194" port: 41163 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:21:51.567178 17123 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:21:51.567463 17123 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47188:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 12:21:51.571128 17123 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e674957562864f0e8c93ce07006a4eea: Sending DeleteTablet for 3 replicas of tablet 1abcd7b4961c44aa80690298f27b0ea3
I20260809 12:21:51.572149 17286 tablet_service.cc:1514] Processing DeleteTablet for tablet 1abcd7b4961c44aa80690298f27b0ea3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:21:51 UTC) from {username='slave'} at 127.0.0.1:46270
I20260809 12:21:51.572431 17471 tablet_service.cc:1514] Processing DeleteTablet for tablet 1abcd7b4961c44aa80690298f27b0ea3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:21:51 UTC) from {username='slave'} at 127.0.0.1:50976
I20260809 12:21:51.572402 17657 tablet_service.cc:1514] Processing DeleteTablet for tablet 1abcd7b4961c44aa80690298f27b0ea3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 12:21:51 UTC) from {username='slave'} at 127.0.0.1:33780
I20260809 12:21:51.574128 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 17204
I20260809 12:21:51.584926 17816 tablet_replica.cc:330] T 1abcd7b4961c44aa80690298f27b0ea3 P 6b4335b687e143bd824a4c2ae02e0f8f: stopping tablet replica
I20260809 12:21:51.585429 17814 tablet_replica.cc:330] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0: stopping tablet replica
I20260809 12:21:51.585983 17814 raft_consensus.cc:2229] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:21:51.586951 17814 raft_consensus.cc:2258] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:21:51.586480 17816 raft_consensus.cc:2229] T 1abcd7b4961c44aa80690298f27b0ea3 P 6b4335b687e143bd824a4c2ae02e0f8f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 12:21:51.587563 17816 raft_consensus.cc:2258] T 1abcd7b4961c44aa80690298f27b0ea3 P 6b4335b687e143bd824a4c2ae02e0f8f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:21:51.590090 17814 ts_tablet_manager.cc:1905] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:21:51.590587 17815 tablet_replica.cc:330] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6: stopping tablet replica
I20260809 12:21:51.590977 17815 raft_consensus.cc:2229] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 [term 1 LEADER]: Raft consensus shutting down.
I20260809 12:21:51.591501 17815 raft_consensus.cc:2258] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 12:21:51.592696 17816 ts_tablet_manager.cc:1905] T 1abcd7b4961c44aa80690298f27b0ea3 P 6b4335b687e143bd824a4c2ae02e0f8f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:21:51.593451 17815 ts_tablet_manager.cc:1905] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 12:21:51.599571 17816 ts_tablet_manager.cc:1918] T 1abcd7b4961c44aa80690298f27b0ea3 P 6b4335b687e143bd824a4c2ae02e0f8f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:21:51.599737 17816 log.cc:1196] T 1abcd7b4961c44aa80690298f27b0ea3 P 6b4335b687e143bd824a4c2ae02e0f8f: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-1/wal/wals/1abcd7b4961c44aa80690298f27b0ea3
I20260809 12:21:51.600298 17816 ts_tablet_manager.cc:1939] T 1abcd7b4961c44aa80690298f27b0ea3 P 6b4335b687e143bd824a4c2ae02e0f8f: Deleting consensus metadata
I20260809 12:21:51.601122 17814 ts_tablet_manager.cc:1918] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:21:51.601284 17814 log.cc:1196] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-0/wal/wals/1abcd7b4961c44aa80690298f27b0ea3
I20260809 12:21:51.601979 17814 ts_tablet_manager.cc:1939] T 1abcd7b4961c44aa80690298f27b0ea3 P 30ae1e28f9044a36ade2966831e457f0: Deleting consensus metadata
I20260809 12:21:51.602366 17104 catalog_manager.cc:4841] TS 6b4335b687e143bd824a4c2ae02e0f8f (127.19.183.194:41163): tablet 1abcd7b4961c44aa80690298f27b0ea3 (table test-table [id=659d5933a74e497483ccf6849d43ffee]) successfully deleted
I20260809 12:21:51.603564 17769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:51.575416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46274 (local address 127.19.183.193:32811)
0809 12:21:51.575640 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:51.575649 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:51.589686 (+ 14037us) server_negotiation.cc:407] Connection header received
0809 12:21:51.589902 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:51.589907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:51.589997 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:51.590109 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:51.592355 (+ 2246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.593435 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.595323 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.595980 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.596076 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:51.600894 (+ 4818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:51.600905 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:51.600921 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:51.601770 (+ 849us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:51.601836 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:51.601866 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:51.601995 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:51.603032 (+ 1037us) server_negotiation.cc:299] Negotiation successful
0809 12:21:51.603174 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 12:21:51.603797 17106 catalog_manager.cc:4841] TS 30ae1e28f9044a36ade2966831e457f0 (127.19.183.193:32811): tablet 1abcd7b4961c44aa80690298f27b0ea3 (table test-table [id=659d5933a74e497483ccf6849d43ffee]) successfully deleted
I20260809 12:21:51.604728 17370 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:51.612106 17815 ts_tablet_manager.cc:1918] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 12:21:51.612293 17815 log.cc:1196] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6: Deleting WAL directory at /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786277774779245-20191-0/minicluster-data/ts-2/wal/wals/1abcd7b4961c44aa80690298f27b0ea3
I20260809 12:21:51.612831 17815 ts_tablet_manager.cc:1939] T 1abcd7b4961c44aa80690298f27b0ea3 P 09b46afc0f4349c7b3ce9741ece0b6b6: Deleting consensus metadata
I20260809 12:21:51.614651 17103 catalog_manager.cc:4841] TS 09b46afc0f4349c7b3ce9741ece0b6b6 (127.19.183.195:42233): tablet 1abcd7b4961c44aa80690298f27b0ea3 (table test-table [id=659d5933a74e497483ccf6849d43ffee]) successfully deleted
I20260809 12:21:51.694373 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 17204
I20260809 12:21:51.729696 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 17379
I20260809 12:21:51.736609 17772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:51.730552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50978 (local address 127.19.183.194:41163)
0809 12:21:51.730797 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:51.730804 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:51.730824 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:21:51.730992 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:51.730996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:51.731081 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:51.731204 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:51.731630 (+ 426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.732378 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.733620 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.734025 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.734165 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:51.735313 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:51.735323 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:51.735338 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:51.735919 (+ 581us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:51.735963 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:51.735981 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:51.736113 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:51.736211 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 12:21:51.736318 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 12:21:51.737238 17550 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:51.826821 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 17379
I20260809 12:21:51.861290 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 17562
I20260809 12:21:51.869197 17775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:51.862204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33800 (local address 127.19.183.195:42233)
0809 12:21:51.862362 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:51.862366 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:51.862384 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:21:51.862534 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:51.862538 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:51.862642 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:51.862796 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:51.863274 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.864031 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.865905 (+ 1874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:51.866338 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:51.866439 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:51.867753 (+ 1314us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:51.867761 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:51.867776 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:51.868442 (+ 666us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:51.868490 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:51.868510 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:51.868589 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:51.868775 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 12:21:51.868854 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 12:21:51.870082 17745 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:51.986853 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 17562
I20260809 12:21:52.017397 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 17084
I20260809 12:21:52.018735 17167 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:52.129127 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 17084
2026-08-09T12:21:52Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (2811 ms)
[----------] 6 tests from EncryptionPolicyTest (23316 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:21:52 dist-test-slave-39l8 krb5kdc[17825](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:21:52 dist-test-slave-39l8 krb5kdc[17825](info): set up 2 sockets
Aug 09 12:21:52 dist-test-slave-39l8 krb5kdc[17825](info): commencing operation
krb5kdc: starting...
W20260809 12:21:53.440414 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.252s user 0.001s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:21:53 dist-test-slave-39l8 krb5kdc[17825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:21:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:21:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:21:53.727218 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40473
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40473
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:21:54.071466 17840 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:54.071765 17840 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:54.071936 17840 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:54.078969 17840 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:21:54.079137 17840 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:54.079223 17840 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:54.079327 17840 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:21:54.079407 17840 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:21:54.089318 17840 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40473
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40473
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:54.090627 17840 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:54.092015 17840 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:54.103811 17849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:54.103960 17854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:54.103883 17851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:54.104626 17840 server_base.cc:1034] running on GCE node
I20260809 12:21:54.105376 17840 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:54.106662 17840 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:54.107916 17840 hybrid_clock.cc:648] HybridClock initialized: now 1786278114107866 us; error 44 us; skew 500 ppm
I20260809 12:21:54.108275 17840 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:21:54 dist-test-slave-39l8 krb5kdc[17825](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278114, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:21:54.111512 17840 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:21:54.112936 17840 webserver.cc:460] Webserver started at http://127.0.0.1:37319/ using document root <none> and password file <none>
I20260809 12:21:54.113633 17840 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:54.113785 17840 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:54.114100 17840 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:54.117091 17840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "4ea39acdc25d407193d863b43acbccb4"
format_stamp: "Formatted at 2026-08-09 12:21:54 on dist-test-slave-39l8"
server_key: "455690c4f35dcc679878805336854bb3"
server_key_iv: "d19aa15cf4fbffeb16cd055ac9ea3d9b"
server_key_version: "encryptionkey@0"
I20260809 12:21:54.117852 17840 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "4ea39acdc25d407193d863b43acbccb4"
format_stamp: "Formatted at 2026-08-09 12:21:54 on dist-test-slave-39l8"
server_key: "455690c4f35dcc679878805336854bb3"
server_key_iv: "d19aa15cf4fbffeb16cd055ac9ea3d9b"
server_key_version: "encryptionkey@0"
I20260809 12:21:54.123087 17840 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.002s
I20260809 12:21:54.126400 17866 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:54.127795 17840 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:21:54.128011 17840 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "4ea39acdc25d407193d863b43acbccb4"
format_stamp: "Formatted at 2026-08-09 12:21:54 on dist-test-slave-39l8"
server_key: "455690c4f35dcc679878805336854bb3"
server_key_iv: "d19aa15cf4fbffeb16cd055ac9ea3d9b"
server_key_version: "encryptionkey@0"
I20260809 12:21:54.128235 17840 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:54.145149 17840 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:54.149349 17840 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:54.149631 17840 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:54.168440 17840 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40473
I20260809 12:21:54.168476 17938 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40473 every 8 connection(s)
I20260809 12:21:54.169788 17840 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:21:54.173697 17939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:54.173697 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 17840
I20260809 12:21:54.173902 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:21:54.174815 20191 external_mini_cluster.cc:1428] Setting key 6f7cbaeed977e64db252aa791caf6199
I20260809 12:21:54.187829 17939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ea39acdc25d407193d863b43acbccb4: Bootstrap starting.
Aug 09 12:21:54 dist-test-slave-39l8 krb5kdc[17825](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278113, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 12:21:54.190907 17939 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ea39acdc25d407193d863b43acbccb4: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:54.192171 17939 log.cc:824] T 00000000000000000000000000000000 P 4ea39acdc25d407193d863b43acbccb4: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:54.195125 17939 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ea39acdc25d407193d863b43acbccb4: No bootstrap required, opened a new log
I20260809 12:21:54.198637 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:54.176753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46279 (local address 127.0.0.1:40473)
0809 12:21:54.177610 (+ 857us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:54.177622 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:54.177665 (+ 43us) server_negotiation.cc:407] Connection header received
0809 12:21:54.180001 (+ 2336us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:54.180024 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:54.180409 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:54.180762 (+ 353us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:54.182740 (+ 1978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:54.184053 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:54.185941 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:54.186539 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:54.186670 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:54.191088 (+ 4418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:54.191098 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:54.191113 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:54.191153 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:54.194378 (+ 3225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:54.195267 (+ 889us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:54.195274 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:54.195280 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:54.195450 (+ 170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:54.195937 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:54.195941 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:54.195943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:54.196439 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:54.196818 (+ 379us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:54.197301 (+ 483us) server_negotiation.cc:299] Negotiation successful
0809 12:21:54.197624 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":649,"thread_start_us":399,"threads_started":1}
I20260809 12:21:54.202790 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:21:54.203917 17939 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ea39acdc25d407193d863b43acbccb4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea39acdc25d407193d863b43acbccb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40473 } }
I20260809 12:21:54.204286 17939 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ea39acdc25d407193d863b43acbccb4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:54.204391 17939 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ea39acdc25d407193d863b43acbccb4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ea39acdc25d407193d863b43acbccb4, State: Initialized, Role: FOLLOWER
I20260809 12:21:54.204993 17939 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ea39acdc25d407193d863b43acbccb4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea39acdc25d407193d863b43acbccb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40473 } }
I20260809 12:21:54.205237 17939 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ea39acdc25d407193d863b43acbccb4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:21:54.205386 17939 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ea39acdc25d407193d863b43acbccb4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:21:54.205571 17939 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ea39acdc25d407193d863b43acbccb4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:54.208364 17939 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ea39acdc25d407193d863b43acbccb4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea39acdc25d407193d863b43acbccb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40473 } }
I20260809 12:21:54.208874 17939 leader_election.cc:302] T 00000000000000000000000000000000 P 4ea39acdc25d407193d863b43acbccb4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ea39acdc25d407193d863b43acbccb4; no voters:
I20260809 12:21:54.209512 17939 leader_election.cc:288] T 00000000000000000000000000000000 P 4ea39acdc25d407193d863b43acbccb4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:21:54.209630 17957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ea39acdc25d407193d863b43acbccb4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:21:54.210899 17957 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ea39acdc25d407193d863b43acbccb4 [term 1 LEADER]: Becoming Leader. State: Replica: 4ea39acdc25d407193d863b43acbccb4, State: Running, Role: LEADER
I20260809 12:21:54.211551 17957 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ea39acdc25d407193d863b43acbccb4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea39acdc25d407193d863b43acbccb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40473 } }
I20260809 12:21:54.212023 17939 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ea39acdc25d407193d863b43acbccb4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:21:54.214558 17959 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ea39acdc25d407193d863b43acbccb4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ea39acdc25d407193d863b43acbccb4. Latest consensus state: current_term: 1 leader_uuid: "4ea39acdc25d407193d863b43acbccb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea39acdc25d407193d863b43acbccb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40473 } } }
I20260809 12:21:54.215025 17959 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ea39acdc25d407193d863b43acbccb4 [sys.catalog]: This master's current role is: LEADER
I20260809 12:21:54.215509 17958 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ea39acdc25d407193d863b43acbccb4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ea39acdc25d407193d863b43acbccb4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea39acdc25d407193d863b43acbccb4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40473 } } }
I20260809 12:21:54.215797 17958 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ea39acdc25d407193d863b43acbccb4 [sys.catalog]: This master's current role is: LEADER
I20260809 12:21:54.227949 17943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:54.207697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49551 (local address 127.0.0.1:40473)
0809 12:21:54.207860 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:54.207864 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:54.208339 (+ 475us) server_negotiation.cc:407] Connection header received
0809 12:21:54.208536 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:54.208543 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:54.208616 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:54.208728 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:54.210617 (+ 1889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:54.211749 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:54.213142 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:54.214271 (+ 1129us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:54.214443 (+ 172us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:54.218636 (+ 4193us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:54.218646 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:54.218654 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:21:54.218720 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:54.223539 (+ 4819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:54.224894 (+ 1355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:54.224903 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:54.224909 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:54.225086 (+ 177us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:21:54.225729 (+ 643us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:21:54.225737 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:21:54.225743 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:21:54.226259 (+ 516us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:54.226654 (+ 395us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:54.227466 (+ 812us) server_negotiation.cc:299] Negotiation successful
0809 12:21:54.227656 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 12:21:54.227979 17963 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:21:54.234764 17963 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:21:54.244732 17963 catalog_manager.cc:1347] Generated new cluster ID: ef20b192b4114904809cdfaff202d45a
I20260809 12:21:54.244858 17963 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:21:54.255728 17963 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:21:54.257571 17963 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:21:54.266575 17963 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ea39acdc25d407193d863b43acbccb4: Generated new TSK 0
I20260809 12:21:54.267364 17963 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:21:54.277016 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 17840
I20260809 12:21:54.278676 17929 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:21:54.366515 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 17840
W20260809 12:21:54.391069 17823 connection.cc:535] client connection to 127.0.0.1:40473 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40473): Connection reset by peer (error 104)
2026-08-09T12:21:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (2222 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:21:54 dist-test-slave-39l8 krb5kdc[17984](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:21:54 dist-test-slave-39l8 krb5kdc[17984](info): set up 2 sockets
Aug 09 12:21:54 dist-test-slave-39l8 krb5kdc[17984](info): commencing operation
krb5kdc: starting...
W20260809 12:21:55.651921 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.000s sys 0.041s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:21:55 dist-test-slave-39l8 krb5kdc[17984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278115, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:21:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:21:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:21:55.951931 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34973
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34973
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:21:56.289129 18001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:56.289390 18001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:56.289475 18001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:56.296402 18001 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:21:56.296501 18001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:56.296546 18001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:56.296607 18001 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:21:56.296655 18001 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:21:56.305572 18001 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34973
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34973
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:56.306759 18001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:56.307987 18001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:56.319820 18008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:56.319880 18011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:56.319875 18009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:56.320807 18001 server_base.cc:1034] running on GCE node
I20260809 12:21:56.321491 18001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:56.322890 18001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:56.324116 18001 hybrid_clock.cc:648] HybridClock initialized: now 1786278116324072 us; error 54 us; skew 500 ppm
I20260809 12:21:56.324491 18001 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:21:56 dist-test-slave-39l8 krb5kdc[17984](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278116, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:21:56.327718 18001 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:21:56.329056 18001 webserver.cc:460] Webserver started at http://127.0.0.1:46719/ using document root <none> and password file <none>
I20260809 12:21:56.329710 18001 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:56.329860 18001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:56.330143 18001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:56.333060 18001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "2338e96a44c04bad88075d43b1db2795"
format_stamp: "Formatted at 2026-08-09 12:21:56 on dist-test-slave-39l8"
server_key: "b6f08bbbf442dba6c36004d1cab4ebba"
server_key_iv: "8d1c7dd8f4692235ac549d2882d02bc5"
server_key_version: "encryptionkey@0"
I20260809 12:21:56.333839 18001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "2338e96a44c04bad88075d43b1db2795"
format_stamp: "Formatted at 2026-08-09 12:21:56 on dist-test-slave-39l8"
server_key: "b6f08bbbf442dba6c36004d1cab4ebba"
server_key_iv: "8d1c7dd8f4692235ac549d2882d02bc5"
server_key_version: "encryptionkey@0"
I20260809 12:21:56.338888 18001 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.008s sys 0.000s
I20260809 12:21:56.342692 18024 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:56.343945 18001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:21:56.344138 18001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "2338e96a44c04bad88075d43b1db2795"
format_stamp: "Formatted at 2026-08-09 12:21:56 on dist-test-slave-39l8"
server_key: "b6f08bbbf442dba6c36004d1cab4ebba"
server_key_iv: "8d1c7dd8f4692235ac549d2882d02bc5"
server_key_version: "encryptionkey@0"
I20260809 12:21:56.344334 18001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:56.372747 18001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:56.377041 18001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:56.377349 18001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:56.395777 18001 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34973
I20260809 12:21:56.395823 18102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34973 every 8 connection(s)
I20260809 12:21:56.397104 18001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:21:56.399806 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 18001
I20260809 12:21:56.399992 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:21:56.400573 20191 external_mini_cluster.cc:1428] Setting key 9cdaa191de68f18ce94a2efbe09ec190
I20260809 12:21:56.401327 18113 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:56.413520 18117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:56.402394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52281 (local address 127.0.0.1:34973)
0809 12:21:56.403182 (+ 788us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:56.403193 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:56.403232 (+ 39us) server_negotiation.cc:407] Connection header received
0809 12:21:56.404721 (+ 1489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:56.404741 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:56.405125 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:56.405451 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:56.406065 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:56.407407 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:56.409215 (+ 1808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:56.409898 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:56.410031 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:56.411349 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:56.411356 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:56.411367 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:56.411863 (+ 496us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:56.411926 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:56.411950 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:56.412100 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:56.412424 (+ 324us) server_negotiation.cc:299] Negotiation successful
0809 12:21:56.412611 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":376,"threads_started":1}
I20260809 12:21:56.414934 18113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2338e96a44c04bad88075d43b1db2795: Bootstrap starting.
I20260809 12:21:56.417277 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:21:56.418268 18113 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2338e96a44c04bad88075d43b1db2795: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:56.419497 18113 log.cc:824] T 00000000000000000000000000000000 P 2338e96a44c04bad88075d43b1db2795: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:56.422154 18113 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2338e96a44c04bad88075d43b1db2795: No bootstrap required, opened a new log
I20260809 12:21:56.429715 18117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:56.422554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39175 (local address 127.0.0.1:34973)
0809 12:21:56.422791 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:56.422799 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:56.423261 (+ 462us) server_negotiation.cc:407] Connection header received
0809 12:21:56.423538 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:56.423542 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:56.423612 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:56.423690 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:56.424304 (+ 614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:56.425262 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:56.427150 (+ 1888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:21:56.427661 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:21:56.427759 (+ 98us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:21:56.428939 (+ 1180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:56.428944 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:56.428947 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:56.428962 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:56.428999 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:56.429012 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:56.429089 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:56.429314 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 12:21:56.429353 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 12:21:56.431742 18113 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2338e96a44c04bad88075d43b1db2795 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2338e96a44c04bad88075d43b1db2795" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34973 } }
I20260809 12:21:56.432158 18113 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2338e96a44c04bad88075d43b1db2795 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:56.432271 18113 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2338e96a44c04bad88075d43b1db2795 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2338e96a44c04bad88075d43b1db2795, State: Initialized, Role: FOLLOWER
I20260809 12:21:56.432801 18113 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2338e96a44c04bad88075d43b1db2795 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2338e96a44c04bad88075d43b1db2795" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34973 } }
I20260809 12:21:56.433040 18113 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2338e96a44c04bad88075d43b1db2795 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:21:56.433189 18113 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2338e96a44c04bad88075d43b1db2795 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:21:56.433364 18113 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2338e96a44c04bad88075d43b1db2795 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:56.435545 18113 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2338e96a44c04bad88075d43b1db2795 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2338e96a44c04bad88075d43b1db2795" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34973 } }
I20260809 12:21:56.436019 18113 leader_election.cc:302] T 00000000000000000000000000000000 P 2338e96a44c04bad88075d43b1db2795 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2338e96a44c04bad88075d43b1db2795; no voters:
I20260809 12:21:56.436600 18113 leader_election.cc:288] T 00000000000000000000000000000000 P 2338e96a44c04bad88075d43b1db2795 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:21:56.436766 18125 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2338e96a44c04bad88075d43b1db2795 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:21:56.438551 18113 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2338e96a44c04bad88075d43b1db2795 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:21:56.438987 18125 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2338e96a44c04bad88075d43b1db2795 [term 1 LEADER]: Becoming Leader. State: Replica: 2338e96a44c04bad88075d43b1db2795, State: Running, Role: LEADER
I20260809 12:21:56.439460 18125 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2338e96a44c04bad88075d43b1db2795 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2338e96a44c04bad88075d43b1db2795" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34973 } }
I20260809 12:21:56.442404 18126 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2338e96a44c04bad88075d43b1db2795 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2338e96a44c04bad88075d43b1db2795" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2338e96a44c04bad88075d43b1db2795" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34973 } } }
I20260809 12:21:56.442484 18127 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2338e96a44c04bad88075d43b1db2795 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2338e96a44c04bad88075d43b1db2795. Latest consensus state: current_term: 1 leader_uuid: "2338e96a44c04bad88075d43b1db2795" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2338e96a44c04bad88075d43b1db2795" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34973 } } }
I20260809 12:21:56.442829 18126 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2338e96a44c04bad88075d43b1db2795 [sys.catalog]: This master's current role is: LEADER
I20260809 12:21:56.442888 18127 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2338e96a44c04bad88075d43b1db2795 [sys.catalog]: This master's current role is: LEADER
I20260809 12:21:56.443892 18144 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:21:56.449150 18144 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:21:56.457434 18144 catalog_manager.cc:1347] Generated new cluster ID: a31b977316664772b9386c77c38b7314
I20260809 12:21:56.457535 18144 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:21:56.476843 18144 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:21:56.478498 18144 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:21:56.484248 18144 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2338e96a44c04bad88075d43b1db2795: Generated new TSK 0
I20260809 12:21:56.484931 18144 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:21:56.490697 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 18001
W20260809 12:21:56.492095 18097 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:52281
W20260809 12:21:56.492923 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 18001: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:21:56.493106 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 18001
W20260809 12:21:56.522360 17982 connection.cc:535] client connection to 127.0.0.1:34973 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34973): Connection reset by peer (error 104)
2026-08-09T12:21:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (2130 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:21:56 dist-test-slave-39l8 krb5kdc[18157](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:21:56 dist-test-slave-39l8 krb5kdc[18157](info): set up 2 sockets
Aug 09 12:21:56 dist-test-slave-39l8 krb5kdc[18157](info): commencing operation
krb5kdc: starting...
W20260809 12:21:57.757472 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.216s user 0.000s sys 0.042s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:21:57 dist-test-slave-39l8 krb5kdc[18157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278117, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:21:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:21:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:21:58.063033 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42855
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42855
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:21:58.401748 18180 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:21:58.402058 18180 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:21:58.402245 18180 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:21:58.409408 18180 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:21:58.409549 18180 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:21:58.409651 18180 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:21:58.409762 18180 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:21:58.409838 18180 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:21:58.419104 18180 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42855
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42855
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:21:58.420285 18180 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:21:58.421515 18180 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:21:58.433126 18187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:58.433178 18189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:21:58.433355 18193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:21:58.433899 18180 server_base.cc:1034] running on GCE node
I20260809 12:21:58.434629 18180 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:21:58.435968 18180 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:21:58.437188 18180 hybrid_clock.cc:648] HybridClock initialized: now 1786278118437136 us; error 35 us; skew 500 ppm
I20260809 12:21:58.437515 18180 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:21:58 dist-test-slave-39l8 krb5kdc[18157](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278118, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:21:58.444295 18180 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:21:58.445673 18180 webserver.cc:460] Webserver started at http://127.0.0.1:42801/ using document root <none> and password file <none>
I20260809 12:21:58.446331 18180 fs_manager.cc:359] Metadata directory not provided
I20260809 12:21:58.446498 18180 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:21:58.446875 18180 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:21:58.449816 18180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "cedd15756f014396889b1875706bd4f1"
format_stamp: "Formatted at 2026-08-09 12:21:58 on dist-test-slave-39l8"
server_key: "fff95250c743d74f8b3b45c48831fd21"
server_key_iv: "25515dfda33b77287d6679e2dcb15887"
server_key_version: "encryptionkey@0"
I20260809 12:21:58.450599 18180 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "cedd15756f014396889b1875706bd4f1"
format_stamp: "Formatted at 2026-08-09 12:21:58 on dist-test-slave-39l8"
server_key: "fff95250c743d74f8b3b45c48831fd21"
server_key_iv: "25515dfda33b77287d6679e2dcb15887"
server_key_version: "encryptionkey@0"
I20260809 12:21:58.455662 18180 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 12:21:58.459080 18203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:21:58.460300 18180 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 12:21:58.460482 18180 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "cedd15756f014396889b1875706bd4f1"
format_stamp: "Formatted at 2026-08-09 12:21:58 on dist-test-slave-39l8"
server_key: "fff95250c743d74f8b3b45c48831fd21"
server_key_iv: "25515dfda33b77287d6679e2dcb15887"
server_key_version: "encryptionkey@0"
I20260809 12:21:58.460680 18180 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:21:58.466043 18180 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:21:58.470208 18180 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:21:58.470471 18180 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:21:58.488384 18180 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42855
I20260809 12:21:58.488461 18279 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42855 every 8 connection(s)
I20260809 12:21:58.489677 18180 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:21:58.490088 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 18180
I20260809 12:21:58.490301 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:21:58.490886 20191 external_mini_cluster.cc:1428] Setting key d5d3787aed69fd65a1116feea21bd70b
I20260809 12:21:58.493940 18280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:21:58.499393 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:58.492809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43417 (local address 127.0.0.1:42855)
0809 12:21:58.493776 (+ 967us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:58.493791 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:58.494414 (+ 623us) server_negotiation.cc:407] Connection header received
0809 12:21:58.496166 (+ 1752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:58.496186 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:58.496491 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:58.496806 (+ 315us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:58.497807 (+ 1001us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:58.497813 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:58.497828 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:58.497842 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:58.497922 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:58.497942 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:58.498108 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:58.498245 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 12:21:58.498448 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":688,"thread_start_us":422,"threads_started":1}
I20260809 12:21:58.503012 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:21:58.507238 18280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cedd15756f014396889b1875706bd4f1: Bootstrap starting.
I20260809 12:21:58.510699 18284 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:21:58.508242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33565 (local address 127.0.0.1:42855)
0809 12:21:58.508417 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 12:21:58.508423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:21:58.508823 (+ 400us) server_negotiation.cc:407] Connection header received
0809 12:21:58.508960 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:21:58.508966 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:21:58.509040 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:21:58.509176 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:21:58.510080 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:21:58.510085 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:21:58.510089 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:21:58.510115 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:21:58.510157 (+ 42us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:21:58.510180 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:21:58.510303 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0809 12:21:58.510371 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 12:21:58.510420 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260809 12:21:58.510939 18280 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P cedd15756f014396889b1875706bd4f1: Neither blocks nor log segments found. Creating new log.
I20260809 12:21:58.512449 18280 log.cc:824] T 00000000000000000000000000000000 P cedd15756f014396889b1875706bd4f1: Log is configured to *not* fsync() on all Append() calls
I20260809 12:21:58.515067 18280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P cedd15756f014396889b1875706bd4f1: No bootstrap required, opened a new log
I20260809 12:21:58.522121 18280 raft_consensus.cc:348] T 00000000000000000000000000000000 P cedd15756f014396889b1875706bd4f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cedd15756f014396889b1875706bd4f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42855 } }
I20260809 12:21:58.522488 18280 raft_consensus.cc:374] T 00000000000000000000000000000000 P cedd15756f014396889b1875706bd4f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:21:58.522590 18280 raft_consensus.cc:729] T 00000000000000000000000000000000 P cedd15756f014396889b1875706bd4f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cedd15756f014396889b1875706bd4f1, State: Initialized, Role: FOLLOWER
I20260809 12:21:58.523247 18280 consensus_queue.cc:260] T 00000000000000000000000000000000 P cedd15756f014396889b1875706bd4f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cedd15756f014396889b1875706bd4f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42855 } }
I20260809 12:21:58.523514 18280 raft_consensus.cc:388] T 00000000000000000000000000000000 P cedd15756f014396889b1875706bd4f1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:21:58.523675 18280 raft_consensus.cc:482] T 00000000000000000000000000000000 P cedd15756f014396889b1875706bd4f1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:21:58.523896 18280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P cedd15756f014396889b1875706bd4f1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:21:58.526572 18280 raft_consensus.cc:504] T 00000000000000000000000000000000 P cedd15756f014396889b1875706bd4f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cedd15756f014396889b1875706bd4f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42855 } }
I20260809 12:21:58.527102 18280 leader_election.cc:302] T 00000000000000000000000000000000 P cedd15756f014396889b1875706bd4f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: cedd15756f014396889b1875706bd4f1; no voters:
I20260809 12:21:58.527684 18280 leader_election.cc:288] T 00000000000000000000000000000000 P cedd15756f014396889b1875706bd4f1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:21:58.527812 18293 raft_consensus.cc:2781] T 00000000000000000000000000000000 P cedd15756f014396889b1875706bd4f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:21:58.528534 18293 raft_consensus.cc:686] T 00000000000000000000000000000000 P cedd15756f014396889b1875706bd4f1 [term 1 LEADER]: Becoming Leader. State: Replica: cedd15756f014396889b1875706bd4f1, State: Running, Role: LEADER
I20260809 12:21:58.529066 18293 consensus_queue.cc:237] T 00000000000000000000000000000000 P cedd15756f014396889b1875706bd4f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cedd15756f014396889b1875706bd4f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42855 } }
I20260809 12:21:58.529711 18280 sys_catalog.cc:564] T 00000000000000000000000000000000 P cedd15756f014396889b1875706bd4f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:21:58.534046 18294 sys_catalog.cc:455] T 00000000000000000000000000000000 P cedd15756f014396889b1875706bd4f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "cedd15756f014396889b1875706bd4f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cedd15756f014396889b1875706bd4f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42855 } } }
I20260809 12:21:58.534492 18294 sys_catalog.cc:458] T 00000000000000000000000000000000 P cedd15756f014396889b1875706bd4f1 [sys.catalog]: This master's current role is: LEADER
I20260809 12:21:58.534794 18295 sys_catalog.cc:455] T 00000000000000000000000000000000 P cedd15756f014396889b1875706bd4f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader cedd15756f014396889b1875706bd4f1. Latest consensus state: current_term: 1 leader_uuid: "cedd15756f014396889b1875706bd4f1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "cedd15756f014396889b1875706bd4f1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42855 } } }
I20260809 12:21:58.535310 18295 sys_catalog.cc:458] T 00000000000000000000000000000000 P cedd15756f014396889b1875706bd4f1 [sys.catalog]: This master's current role is: LEADER
I20260809 12:21:58.536438 18305 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:21:58.541298 18305 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:21:58.548911 18305 catalog_manager.cc:1347] Generated new cluster ID: 1a191fdc149f4d7fb42c08ea11b72444
I20260809 12:21:58.549022 18305 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:21:58.590246 18305 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:21:58.590773 18305 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:21:58.601289 18305 catalog_manager.cc:5868] T 00000000000000000000000000000000 P cedd15756f014396889b1875706bd4f1: Generated new TSK 0
I20260809 12:21:58.602207 18305 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:21:58.610078 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 18180
W20260809 12:21:58.611188 18269 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:43417
W20260809 12:21:58.611855 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 18180: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:21:58.612025 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 18180
2026-08-09T12:21:58Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (2120 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:21:58 dist-test-slave-39l8 krb5kdc[18323](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:21:58 dist-test-slave-39l8 krb5kdc[18323](info): set up 2 sockets
Aug 09 12:21:58 dist-test-slave-39l8 krb5kdc[18323](info): commencing operation
krb5kdc: starting...
W20260809 12:21:59.918630 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.000s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:22:00 dist-test-slave-39l8 krb5kdc[18323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:22:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:22:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:22:00.245743 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41087
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44429
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41087
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:22:00.579310 18339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:22:00.579623 18339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:22:00.579785 18339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:22:00.586603 18339 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:22:00.586798 18339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:22:00.586927 18339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:22:00.587046 18339 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:22:00.587123 18339 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:22:00.595899 18339 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44429
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41087
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41087
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:22:00.597095 18339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:22:00.598332 18339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:22:00.609869 18347 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:00.609822 18348 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:00.609907 18352 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:22:00.610754 18339 server_base.cc:1034] running on GCE node
I20260809 12:22:00.611516 18339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:22:00.612843 18339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:22:00.614073 18339 hybrid_clock.cc:648] HybridClock initialized: now 1786278120614037 us; error 46 us; skew 500 ppm
I20260809 12:22:00.614416 18339 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:22:00 dist-test-slave-39l8 krb5kdc[18323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278120, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:22:00.617479 18339 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:22:00.618988 18339 webserver.cc:460] Webserver started at http://127.0.0.1:38031/ using document root <none> and password file <none>
I20260809 12:22:00.619676 18339 fs_manager.cc:359] Metadata directory not provided
I20260809 12:22:00.619827 18339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:22:00.620133 18339 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:22:00.623072 18339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "54b12c4b8f5048dbb49680c0d30045b2"
format_stamp: "Formatted at 2026-08-09 12:22:00 on dist-test-slave-39l8"
server_key: "56f9956c4d3a9552fda253bde8989b58"
server_key_iv: "e17c34ed624cd717db142bda13f6e39f"
server_key_version: "encryptionkey@0"
I20260809 12:22:00.623843 18339 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "54b12c4b8f5048dbb49680c0d30045b2"
format_stamp: "Formatted at 2026-08-09 12:22:00 on dist-test-slave-39l8"
server_key: "56f9956c4d3a9552fda253bde8989b58"
server_key_iv: "e17c34ed624cd717db142bda13f6e39f"
server_key_version: "encryptionkey@0"
I20260809 12:22:00.629015 18339 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 12:22:00.632313 18365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:22:00.633591 18339 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:22:00.633800 18339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "54b12c4b8f5048dbb49680c0d30045b2"
format_stamp: "Formatted at 2026-08-09 12:22:00 on dist-test-slave-39l8"
server_key: "56f9956c4d3a9552fda253bde8989b58"
server_key_iv: "e17c34ed624cd717db142bda13f6e39f"
server_key_version: "encryptionkey@0"
I20260809 12:22:00.633980 18339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:22:00.671234 18339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:22:00.675259 18339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:22:00.675581 18339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:22:00.694301 18339 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41087
I20260809 12:22:00.694340 18433 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41087 every 8 connection(s)
I20260809 12:22:00.695760 18339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:22:00.697391 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 18339
I20260809 12:22:00.697603 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:22:00.698156 20191 external_mini_cluster.cc:1428] Setting key 7cd3bf466710bf78d7887997c2b2b172
I20260809 12:22:00.700495 18434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 09 12:22:00 dist-test-slave-39l8 krb5kdc[18323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278120, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 12:22:00.713692 18434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54b12c4b8f5048dbb49680c0d30045b2: Bootstrap starting.
I20260809 12:22:00.716888 18434 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54b12c4b8f5048dbb49680c0d30045b2: Neither blocks nor log segments found. Creating new log.
I20260809 12:22:00.717933 18434 log.cc:824] T 00000000000000000000000000000000 P 54b12c4b8f5048dbb49680c0d30045b2: Log is configured to *not* fsync() on all Append() calls
I20260809 12:22:00.720917 18434 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54b12c4b8f5048dbb49680c0d30045b2: No bootstrap required, opened a new log
I20260809 12:22:00.721154 18438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:00.700028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43421 (local address 127.0.0.1:41087)
0809 12:22:00.700887 (+ 859us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:00.700899 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:00.700940 (+ 41us) server_negotiation.cc:407] Connection header received
0809 12:22:00.702510 (+ 1570us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:00.702532 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:00.702987 (+ 455us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:00.703317 (+ 330us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:00.705110 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:00.706452 (+ 1342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:00.708225 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:00.708837 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:00.708963 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:00.713845 (+ 4882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:00.713857 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:00.713878 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:00.713920 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:00.717067 (+ 3147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:00.717795 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:00.717810 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:00.717816 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:00.717976 (+ 160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:00.718398 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:00.718403 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:00.718406 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:00.719033 (+ 627us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:00.719354 (+ 321us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:00.719803 (+ 449us) server_negotiation.cc:299] Negotiation successful
0809 12:22:00.720214 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":645,"thread_start_us":387,"threads_started":1}
I20260809 12:22:00.724800 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:22:00.728760 18434 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54b12c4b8f5048dbb49680c0d30045b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54b12c4b8f5048dbb49680c0d30045b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41087 } }
I20260809 12:22:00.729192 18434 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54b12c4b8f5048dbb49680c0d30045b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:22:00.729310 18434 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54b12c4b8f5048dbb49680c0d30045b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54b12c4b8f5048dbb49680c0d30045b2, State: Initialized, Role: FOLLOWER
I20260809 12:22:00.729962 18434 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54b12c4b8f5048dbb49680c0d30045b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54b12c4b8f5048dbb49680c0d30045b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41087 } }
I20260809 12:22:00.730255 18434 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54b12c4b8f5048dbb49680c0d30045b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:22:00.730448 18434 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54b12c4b8f5048dbb49680c0d30045b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:22:00.730630 18434 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54b12c4b8f5048dbb49680c0d30045b2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:22:00.733400 18434 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54b12c4b8f5048dbb49680c0d30045b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54b12c4b8f5048dbb49680c0d30045b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41087 } }
I20260809 12:22:00.733884 18434 leader_election.cc:302] T 00000000000000000000000000000000 P 54b12c4b8f5048dbb49680c0d30045b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54b12c4b8f5048dbb49680c0d30045b2; no voters:
I20260809 12:22:00.734552 18434 leader_election.cc:288] T 00000000000000000000000000000000 P 54b12c4b8f5048dbb49680c0d30045b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:22:00.734661 18448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54b12c4b8f5048dbb49680c0d30045b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:22:00.735567 18448 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54b12c4b8f5048dbb49680c0d30045b2 [term 1 LEADER]: Becoming Leader. State: Replica: 54b12c4b8f5048dbb49680c0d30045b2, State: Running, Role: LEADER
I20260809 12:22:00.736092 18448 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54b12c4b8f5048dbb49680c0d30045b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54b12c4b8f5048dbb49680c0d30045b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41087 } }
I20260809 12:22:00.737725 18434 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54b12c4b8f5048dbb49680c0d30045b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:22:00.738991 18449 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54b12c4b8f5048dbb49680c0d30045b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54b12c4b8f5048dbb49680c0d30045b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54b12c4b8f5048dbb49680c0d30045b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41087 } } }
I20260809 12:22:00.739537 18449 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54b12c4b8f5048dbb49680c0d30045b2 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:00.739491 18450 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54b12c4b8f5048dbb49680c0d30045b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54b12c4b8f5048dbb49680c0d30045b2. Latest consensus state: current_term: 1 leader_uuid: "54b12c4b8f5048dbb49680c0d30045b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54b12c4b8f5048dbb49680c0d30045b2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41087 } } }
I20260809 12:22:00.739883 18450 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54b12c4b8f5048dbb49680c0d30045b2 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:00.743634 18455 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:22:00.747476 18438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:00.729139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35499 (local address 127.0.0.1:41087)
0809 12:22:00.729323 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:00.729330 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:00.729853 (+ 523us) server_negotiation.cc:407] Connection header received
0809 12:22:00.729971 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:00.729979 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:00.730057 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:00.730163 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:00.731771 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:00.732956 (+ 1185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:00.734396 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:00.735500 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:00.735672 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:00.739057 (+ 3385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:00.739071 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:00.739077 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:00.739122 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:00.743603 (+ 4481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:00.744272 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:00.744278 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:00.744282 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:00.744533 (+ 251us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:00.745013 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:00.745016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:00.745018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:00.745491 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:00.745858 (+ 367us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:00.746891 (+ 1033us) server_negotiation.cc:299] Negotiation successful
0809 12:22:00.747106 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 12:22:00.750638 18455 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:22:00.760466 18455 catalog_manager.cc:1347] Generated new cluster ID: d08f2a3644af4010bed1e00fc00e0604
I20260809 12:22:00.760583 18455 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:22:00.775840 18455 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:22:00.777642 18455 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:22:00.797714 18455 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54b12c4b8f5048dbb49680c0d30045b2: Generated new TSK 0
I20260809 12:22:00.798544 18455 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:22:00.806478 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 18339
I20260809 12:22:00.807827 18428 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:22:00.899030 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 18339
2026-08-09T12:22:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (2278 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:22:01 dist-test-slave-39l8 krb5kdc[18487](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:22:01 dist-test-slave-39l8 krb5kdc[18487](info): set up 2 sockets
Aug 09 12:22:01 dist-test-slave-39l8 krb5kdc[18487](info): commencing operation
krb5kdc: starting...
W20260809 12:22:02.181754 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.242s user 0.001s sys 0.046s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:22:02 dist-test-slave-39l8 krb5kdc[18487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278122, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:22:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:22:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:22:02.497470 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37359
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32833
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37359
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:22:02.829123 18502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:22:02.829391 18502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:22:02.829547 18502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:22:02.836241 18502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:22:02.836370 18502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:22:02.836479 18502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:22:02.836591 18502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:22:02.836671 18502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:22:02.846110 18502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32833
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37359
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37359
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:22:02.847445 18502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:22:02.848740 18502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:22:02.860129 18516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:02.860088 18514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:02.860163 18513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:22:02.860980 18502 server_base.cc:1034] running on GCE node
I20260809 12:22:02.861807 18502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:22:02.863549 18502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:22:02.864784 18502 hybrid_clock.cc:648] HybridClock initialized: now 1786278122864725 us; error 47 us; skew 500 ppm
I20260809 12:22:02.865211 18502 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:22:02 dist-test-slave-39l8 krb5kdc[18487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278122, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:22:02.868323 18502 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:22:02.869685 18502 webserver.cc:460] Webserver started at http://127.0.0.1:42339/ using document root <none> and password file <none>
I20260809 12:22:02.870357 18502 fs_manager.cc:359] Metadata directory not provided
I20260809 12:22:02.870505 18502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:22:02.870889 18502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:22:02.873900 18502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "591e7b3d11334b40a0d8fc488dd95bb3"
format_stamp: "Formatted at 2026-08-09 12:22:02 on dist-test-slave-39l8"
server_key: "d40fa26b692929caa06a225bfba9d5eb"
server_key_iv: "29419eb7157f6402ea82c751604cd6a4"
server_key_version: "encryptionkey@0"
I20260809 12:22:02.874668 18502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "591e7b3d11334b40a0d8fc488dd95bb3"
format_stamp: "Formatted at 2026-08-09 12:22:02 on dist-test-slave-39l8"
server_key: "d40fa26b692929caa06a225bfba9d5eb"
server_key_iv: "29419eb7157f6402ea82c751604cd6a4"
server_key_version: "encryptionkey@0"
I20260809 12:22:02.879769 18502 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260809 12:22:02.883195 18528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:22:02.884403 18502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 12:22:02.884585 18502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "591e7b3d11334b40a0d8fc488dd95bb3"
format_stamp: "Formatted at 2026-08-09 12:22:02 on dist-test-slave-39l8"
server_key: "d40fa26b692929caa06a225bfba9d5eb"
server_key_iv: "29419eb7157f6402ea82c751604cd6a4"
server_key_version: "encryptionkey@0"
I20260809 12:22:02.884790 18502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:22:02.898041 18502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:22:02.901748 18502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:22:02.902036 18502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:22:02.920665 18502 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37359
I20260809 12:22:02.920702 18599 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37359 every 8 connection(s)
I20260809 12:22:02.922112 18502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:22:02.926044 18600 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:22:02.927075 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 18502
I20260809 12:22:02.927276 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:22:02.927855 20191 external_mini_cluster.cc:1428] Setting key fe258841430303e08a400871d183ffc1
I20260809 12:22:02.939008 18600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 591e7b3d11334b40a0d8fc488dd95bb3: Bootstrap starting.
I20260809 12:22:02.941298 18604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:02.929758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47213 (local address 127.0.0.1:37359)
0809 12:22:02.930472 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:02.930484 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:02.930526 (+ 42us) server_negotiation.cc:407] Connection header received
0809 12:22:02.932727 (+ 2201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:02.932750 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:02.933144 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:02.933541 (+ 397us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:02.934062 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:02.935347 (+ 1285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:02.936669 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:02.937357 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:02.937552 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:02.938874 (+ 1322us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:02.938882 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:02.938896 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:22:02.939511 (+ 615us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:02.939671 (+ 160us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:22:02.939719 (+ 48us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:02.939857 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:02.940111 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 12:22:02.940366 (+ 255us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":504,"thread_start_us":306,"threads_started":1}
I20260809 12:22:02.942459 18600 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 591e7b3d11334b40a0d8fc488dd95bb3: Neither blocks nor log segments found. Creating new log.
I20260809 12:22:02.943959 18600 log.cc:824] T 00000000000000000000000000000000 P 591e7b3d11334b40a0d8fc488dd95bb3: Log is configured to *not* fsync() on all Append() calls
I20260809 12:22:02.945348 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:22:02.946856 18600 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 591e7b3d11334b40a0d8fc488dd95bb3: No bootstrap required, opened a new log
I20260809 12:22:02.955204 18600 raft_consensus.cc:348] T 00000000000000000000000000000000 P 591e7b3d11334b40a0d8fc488dd95bb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "591e7b3d11334b40a0d8fc488dd95bb3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37359 } }
I20260809 12:22:02.955597 18600 raft_consensus.cc:374] T 00000000000000000000000000000000 P 591e7b3d11334b40a0d8fc488dd95bb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:22:02.955741 18600 raft_consensus.cc:729] T 00000000000000000000000000000000 P 591e7b3d11334b40a0d8fc488dd95bb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 591e7b3d11334b40a0d8fc488dd95bb3, State: Initialized, Role: FOLLOWER
I20260809 12:22:02.956257 18604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:02.949830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54273 (local address 127.0.0.1:37359)
0809 12:22:02.950094 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:02.950100 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:02.950420 (+ 320us) server_negotiation.cc:407] Connection header received
0809 12:22:02.950680 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:02.950684 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:02.950827 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:02.950932 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:02.951529 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:02.952352 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:02.953651 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:02.954128 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:02.954285 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:02.955433 (+ 1148us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:02.955437 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:02.955441 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:22:02.955457 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:02.955512 (+ 55us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:22:02.955533 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:02.955653 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:02.955832 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 12:22:02.955924 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 12:22:02.956430 18600 consensus_queue.cc:260] T 00000000000000000000000000000000 P 591e7b3d11334b40a0d8fc488dd95bb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "591e7b3d11334b40a0d8fc488dd95bb3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37359 } }
I20260809 12:22:02.956717 18600 raft_consensus.cc:388] T 00000000000000000000000000000000 P 591e7b3d11334b40a0d8fc488dd95bb3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:22:02.956875 18600 raft_consensus.cc:482] T 00000000000000000000000000000000 P 591e7b3d11334b40a0d8fc488dd95bb3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:22:02.957079 18600 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 591e7b3d11334b40a0d8fc488dd95bb3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:22:02.959544 18600 raft_consensus.cc:504] T 00000000000000000000000000000000 P 591e7b3d11334b40a0d8fc488dd95bb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "591e7b3d11334b40a0d8fc488dd95bb3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37359 } }
I20260809 12:22:02.960130 18600 leader_election.cc:302] T 00000000000000000000000000000000 P 591e7b3d11334b40a0d8fc488dd95bb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 591e7b3d11334b40a0d8fc488dd95bb3; no voters:
I20260809 12:22:02.960646 18600 leader_election.cc:288] T 00000000000000000000000000000000 P 591e7b3d11334b40a0d8fc488dd95bb3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:22:02.960821 18612 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 591e7b3d11334b40a0d8fc488dd95bb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:22:02.961536 18612 raft_consensus.cc:686] T 00000000000000000000000000000000 P 591e7b3d11334b40a0d8fc488dd95bb3 [term 1 LEADER]: Becoming Leader. State: Replica: 591e7b3d11334b40a0d8fc488dd95bb3, State: Running, Role: LEADER
I20260809 12:22:02.962172 18612 consensus_queue.cc:237] T 00000000000000000000000000000000 P 591e7b3d11334b40a0d8fc488dd95bb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "591e7b3d11334b40a0d8fc488dd95bb3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37359 } }
I20260809 12:22:02.962827 18600 sys_catalog.cc:564] T 00000000000000000000000000000000 P 591e7b3d11334b40a0d8fc488dd95bb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:22:02.965055 18614 sys_catalog.cc:455] T 00000000000000000000000000000000 P 591e7b3d11334b40a0d8fc488dd95bb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "591e7b3d11334b40a0d8fc488dd95bb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "591e7b3d11334b40a0d8fc488dd95bb3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37359 } } }
I20260809 12:22:02.965456 18614 sys_catalog.cc:458] T 00000000000000000000000000000000 P 591e7b3d11334b40a0d8fc488dd95bb3 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:02.965972 18615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 591e7b3d11334b40a0d8fc488dd95bb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 591e7b3d11334b40a0d8fc488dd95bb3. Latest consensus state: current_term: 1 leader_uuid: "591e7b3d11334b40a0d8fc488dd95bb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "591e7b3d11334b40a0d8fc488dd95bb3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37359 } } }
I20260809 12:22:02.966298 18615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 591e7b3d11334b40a0d8fc488dd95bb3 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:02.967308 18624 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:22:02.973762 18624 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:22:02.983451 18624 catalog_manager.cc:1347] Generated new cluster ID: 970d0bd806bb4de8a082ad9ac1a47271
I20260809 12:22:02.983580 18624 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:22:03.032506 18624 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:22:03.034489 18624 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:22:03.044508 18624 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 591e7b3d11334b40a0d8fc488dd95bb3: Generated new TSK 0
I20260809 12:22:03.045444 18624 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:22:03.051280 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 18502
W20260809 12:22:03.052402 18595 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:47213
W20260809 12:22:03.053238 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 18502: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:22:03.053431 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 18502
2026-08-09T12:22:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (2161 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:22:03 dist-test-slave-39l8 krb5kdc[18645](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:22:03 dist-test-slave-39l8 krb5kdc[18645](info): set up 2 sockets
Aug 09 12:22:03 dist-test-slave-39l8 krb5kdc[18645](info): commencing operation
krb5kdc: starting...
W20260809 12:22:04.368223 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.266s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:22:04 dist-test-slave-39l8 krb5kdc[18645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278124, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:22:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:22:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:22:04.694057 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35295
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:22:05.038419 18661 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:22:05.038720 18661 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:22:05.038941 18661 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:22:05.046368 18661 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:22:05.046491 18661 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:22:05.046607 18661 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:22:05.046743 18661 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:22:05.046810 18661 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:22:05.057260 18661 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:22:05.058673 18661 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:22:05.060227 18661 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:22:05.072623 18668 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:05.072631 18669 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:05.072566 18673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:22:05.073578 18661 server_base.cc:1034] running on GCE node
I20260809 12:22:05.074405 18661 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:22:05.075914 18661 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:22:05.077165 18661 hybrid_clock.cc:648] HybridClock initialized: now 1786278125077124 us; error 53 us; skew 500 ppm
I20260809 12:22:05.077534 18661 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:22:05 dist-test-slave-39l8 krb5kdc[18645](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278125, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:22:05.085418 18661 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:22:05.087085 18661 webserver.cc:460] Webserver started at http://127.0.0.1:46171/ using document root <none> and password file <none>
I20260809 12:22:05.087841 18661 fs_manager.cc:359] Metadata directory not provided
I20260809 12:22:05.087990 18661 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:22:05.088328 18661 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:22:05.091579 18661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "fb0991fa1d6240219c640482e63444a2"
format_stamp: "Formatted at 2026-08-09 12:22:05 on dist-test-slave-39l8"
server_key: "1abe9642f013603c03fdf3f3268b2c53"
server_key_iv: "b216ed0c27c39835d0971945c7e79d13"
server_key_version: "encryptionkey@0"
I20260809 12:22:05.092419 18661 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "fb0991fa1d6240219c640482e63444a2"
format_stamp: "Formatted at 2026-08-09 12:22:05 on dist-test-slave-39l8"
server_key: "1abe9642f013603c03fdf3f3268b2c53"
server_key_iv: "b216ed0c27c39835d0971945c7e79d13"
server_key_version: "encryptionkey@0"
I20260809 12:22:05.097730 18661 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:22:05.101266 18682 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:22:05.102561 18661 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:22:05.102766 18661 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "fb0991fa1d6240219c640482e63444a2"
format_stamp: "Formatted at 2026-08-09 12:22:05 on dist-test-slave-39l8"
server_key: "1abe9642f013603c03fdf3f3268b2c53"
server_key_iv: "b216ed0c27c39835d0971945c7e79d13"
server_key_version: "encryptionkey@0"
I20260809 12:22:05.102973 18661 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:22:05.108783 18661 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:22:05.113134 18661 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:22:05.113435 18661 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:22:05.131690 18661 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35295
I20260809 12:22:05.131742 18760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35295 every 8 connection(s)
I20260809 12:22:05.133160 18661 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:22:05.136062 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 18661
I20260809 12:22:05.136253 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:22:05.136737 20191 external_mini_cluster.cc:1428] Setting key 3094bc68da394a1629d7d9d90ca10679
I20260809 12:22:05.137054 18762 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:22:05.146652 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:05.138581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55765 (local address 127.0.0.1:35295)
0809 12:22:05.139401 (+ 820us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:05.139413 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:05.139453 (+ 40us) server_negotiation.cc:407] Connection header received
0809 12:22:05.141638 (+ 2185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:05.141659 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:05.141974 (+ 315us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:05.142390 (+ 416us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:05.144970 (+ 2580us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:05.144979 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:05.144995 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:22:05.145017 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:05.145119 (+ 102us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:22:05.145158 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:05.145386 (+ 228us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:05.145518 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 12:22:05.145713 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":606,"thread_start_us":336,"threads_started":1}
I20260809 12:22:05.150599 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:22:05.150950 18762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb0991fa1d6240219c640482e63444a2: Bootstrap starting.
I20260809 12:22:05.154161 18762 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fb0991fa1d6240219c640482e63444a2: Neither blocks nor log segments found. Creating new log.
I20260809 12:22:05.155458 18762 log.cc:824] T 00000000000000000000000000000000 P fb0991fa1d6240219c640482e63444a2: Log is configured to *not* fsync() on all Append() calls
I20260809 12:22:05.158088 18765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:05.155178 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39679 (local address 127.0.0.1:35295)
0809 12:22:05.155419 (+ 241us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:05.155428 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:05.156033 (+ 605us) server_negotiation.cc:407] Connection header received
0809 12:22:05.156213 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:05.156217 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:05.156266 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:05.156346 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:05.157411 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:05.157417 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:05.157421 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:22:05.157439 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:05.157484 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:22:05.157507 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:05.157628 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:05.157741 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 12:22:05.157808 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 12:22:05.158857 18762 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fb0991fa1d6240219c640482e63444a2: No bootstrap required, opened a new log
I20260809 12:22:05.166788 18762 raft_consensus.cc:348] T 00000000000000000000000000000000 P fb0991fa1d6240219c640482e63444a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb0991fa1d6240219c640482e63444a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35295 } }
I20260809 12:22:05.167198 18762 raft_consensus.cc:374] T 00000000000000000000000000000000 P fb0991fa1d6240219c640482e63444a2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:22:05.167315 18762 raft_consensus.cc:729] T 00000000000000000000000000000000 P fb0991fa1d6240219c640482e63444a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb0991fa1d6240219c640482e63444a2, State: Initialized, Role: FOLLOWER
I20260809 12:22:05.167958 18762 consensus_queue.cc:260] T 00000000000000000000000000000000 P fb0991fa1d6240219c640482e63444a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb0991fa1d6240219c640482e63444a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35295 } }
I20260809 12:22:05.168215 18762 raft_consensus.cc:388] T 00000000000000000000000000000000 P fb0991fa1d6240219c640482e63444a2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:22:05.168370 18762 raft_consensus.cc:482] T 00000000000000000000000000000000 P fb0991fa1d6240219c640482e63444a2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:22:05.168545 18762 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fb0991fa1d6240219c640482e63444a2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:22:05.170924 18762 raft_consensus.cc:504] T 00000000000000000000000000000000 P fb0991fa1d6240219c640482e63444a2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb0991fa1d6240219c640482e63444a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35295 } }
I20260809 12:22:05.171427 18762 leader_election.cc:302] T 00000000000000000000000000000000 P fb0991fa1d6240219c640482e63444a2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fb0991fa1d6240219c640482e63444a2; no voters:
I20260809 12:22:05.172013 18762 leader_election.cc:288] T 00000000000000000000000000000000 P fb0991fa1d6240219c640482e63444a2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:22:05.172183 18774 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fb0991fa1d6240219c640482e63444a2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:22:05.172865 18774 raft_consensus.cc:686] T 00000000000000000000000000000000 P fb0991fa1d6240219c640482e63444a2 [term 1 LEADER]: Becoming Leader. State: Replica: fb0991fa1d6240219c640482e63444a2, State: Running, Role: LEADER
I20260809 12:22:05.173455 18774 consensus_queue.cc:237] T 00000000000000000000000000000000 P fb0991fa1d6240219c640482e63444a2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb0991fa1d6240219c640482e63444a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35295 } }
I20260809 12:22:05.174165 18762 sys_catalog.cc:564] T 00000000000000000000000000000000 P fb0991fa1d6240219c640482e63444a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:22:05.176630 18776 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb0991fa1d6240219c640482e63444a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fb0991fa1d6240219c640482e63444a2. Latest consensus state: current_term: 1 leader_uuid: "fb0991fa1d6240219c640482e63444a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb0991fa1d6240219c640482e63444a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35295 } } }
I20260809 12:22:05.176705 18775 sys_catalog.cc:455] T 00000000000000000000000000000000 P fb0991fa1d6240219c640482e63444a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fb0991fa1d6240219c640482e63444a2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fb0991fa1d6240219c640482e63444a2" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35295 } } }
I20260809 12:22:05.177035 18776 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb0991fa1d6240219c640482e63444a2 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:05.177090 18775 sys_catalog.cc:458] T 00000000000000000000000000000000 P fb0991fa1d6240219c640482e63444a2 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:05.179247 18781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:22:05.185226 18781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:22:05.193759 18781 catalog_manager.cc:1347] Generated new cluster ID: d5e6625361674cd99b22c9f80c05eefb
I20260809 12:22:05.193868 18781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:22:05.214428 18781 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:22:05.214937 18781 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:22:05.225214 18781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fb0991fa1d6240219c640482e63444a2: Generated new TSK 0
I20260809 12:22:05.226189 18781 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:22:05.236274 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 18661
W20260809 12:22:05.237866 18755 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:55765
W20260809 12:22:05.238605 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 18661: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:22:05.238790 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 18661
2026-08-09T12:22:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (2183 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:22:05 dist-test-slave-39l8 krb5kdc[18812](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:22:05 dist-test-slave-39l8 krb5kdc[18812](info): set up 2 sockets
Aug 09 12:22:05 dist-test-slave-39l8 krb5kdc[18812](info): commencing operation
krb5kdc: starting...
W20260809 12:22:06.557813 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.272s user 0.003s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:22:06 dist-test-slave-39l8 krb5kdc[18812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:22:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:22:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:22:06.893759 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37941
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:22:07.243052 18829 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:22:07.243414 18829 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:22:07.243577 18829 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:22:07.251029 18829 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:22:07.251194 18829 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:22:07.251297 18829 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:22:07.251405 18829 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:22:07.251492 18829 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:22:07.261268 18829 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:22:07.262540 18829 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:22:07.263944 18829 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:22:07.275921 18836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:07.275949 18840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:07.275990 18837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:22:07.277079 18829 server_base.cc:1034] running on GCE node
I20260809 12:22:07.277885 18829 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:22:07.279400 18829 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:22:07.280627 18829 hybrid_clock.cc:648] HybridClock initialized: now 1786278127280593 us; error 43 us; skew 500 ppm
I20260809 12:22:07.280992 18829 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:22:07 dist-test-slave-39l8 krb5kdc[18812](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278127, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:22:07.284225 18829 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:22:07.285624 18829 webserver.cc:460] Webserver started at http://127.0.0.1:45777/ using document root <none> and password file <none>
I20260809 12:22:07.286345 18829 fs_manager.cc:359] Metadata directory not provided
I20260809 12:22:07.286497 18829 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:22:07.286867 18829 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:22:07.289844 18829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "4a061c25187f4add91c221ec71a9d76c"
format_stamp: "Formatted at 2026-08-09 12:22:07 on dist-test-slave-39l8"
server_key: "0171ee2f31695721271d85990114ff23"
server_key_iv: "cc74809ae70865d43c9e98d99333aa99"
server_key_version: "encryptionkey@0"
I20260809 12:22:07.290624 18829 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "4a061c25187f4add91c221ec71a9d76c"
format_stamp: "Formatted at 2026-08-09 12:22:07 on dist-test-slave-39l8"
server_key: "0171ee2f31695721271d85990114ff23"
server_key_iv: "cc74809ae70865d43c9e98d99333aa99"
server_key_version: "encryptionkey@0"
I20260809 12:22:07.295706 18829 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260809 12:22:07.299103 18852 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:22:07.300307 18829 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.002s
I20260809 12:22:07.300493 18829 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "4a061c25187f4add91c221ec71a9d76c"
format_stamp: "Formatted at 2026-08-09 12:22:07 on dist-test-slave-39l8"
server_key: "0171ee2f31695721271d85990114ff23"
server_key_iv: "cc74809ae70865d43c9e98d99333aa99"
server_key_version: "encryptionkey@0"
I20260809 12:22:07.300691 18829 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:22:07.317536 18829 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:22:07.322117 18829 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:22:07.322445 18829 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:22:07.341528 18829 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37941
I20260809 12:22:07.341550 18923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37941 every 8 connection(s)
I20260809 12:22:07.343042 18829 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:22:07.346896 18924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:22:07.348071 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 18829
I20260809 12:22:07.348269 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:22:07.348897 20191 external_mini_cluster.cc:1428] Setting key 2b5bc4051b437d0b0d37afb32b3ed509
I20260809 12:22:07.359442 18924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a061c25187f4add91c221ec71a9d76c: Bootstrap starting.
Aug 09 12:22:07 dist-test-slave-39l8 krb5kdc[18812](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278126, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 12:22:07.362891 18924 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a061c25187f4add91c221ec71a9d76c: Neither blocks nor log segments found. Creating new log.
I20260809 12:22:07.364202 18924 log.cc:824] T 00000000000000000000000000000000 P 4a061c25187f4add91c221ec71a9d76c: Log is configured to *not* fsync() on all Append() calls
I20260809 12:22:07.367386 18924 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a061c25187f4add91c221ec71a9d76c: No bootstrap required, opened a new log
I20260809 12:22:07.371785 18929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:07.350967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:57777 (local address 127.0.0.1:37941)
0809 12:22:07.351694 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:07.351705 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:07.351751 (+ 46us) server_negotiation.cc:407] Connection header received
0809 12:22:07.353425 (+ 1674us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:07.353447 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:07.353819 (+ 372us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:07.354230 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:07.355672 (+ 1442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:07.357002 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:07.358297 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:07.359053 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:07.359266 (+ 213us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:07.364030 (+ 4764us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:07.364038 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:07.364051 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:07.364086 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:07.367570 (+ 3484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:07.368527 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:07.368533 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:07.368539 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:07.368695 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:07.369219 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:07.369222 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:07.369224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:07.369720 (+ 496us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:07.369961 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:07.370430 (+ 469us) server_negotiation.cc:299] Negotiation successful
0809 12:22:07.370773 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":498,"thread_start_us":297,"threads_started":1}
I20260809 12:22:07.376001 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:22:07.377013 18924 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a061c25187f4add91c221ec71a9d76c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a061c25187f4add91c221ec71a9d76c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37941 } }
I20260809 12:22:07.377445 18924 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a061c25187f4add91c221ec71a9d76c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:22:07.377542 18924 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a061c25187f4add91c221ec71a9d76c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a061c25187f4add91c221ec71a9d76c, State: Initialized, Role: FOLLOWER
I20260809 12:22:07.378129 18924 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a061c25187f4add91c221ec71a9d76c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a061c25187f4add91c221ec71a9d76c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37941 } }
I20260809 12:22:07.378413 18924 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4a061c25187f4add91c221ec71a9d76c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:22:07.378563 18924 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a061c25187f4add91c221ec71a9d76c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:22:07.378782 18924 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a061c25187f4add91c221ec71a9d76c [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:22:07.381891 18924 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a061c25187f4add91c221ec71a9d76c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a061c25187f4add91c221ec71a9d76c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37941 } }
I20260809 12:22:07.382524 18924 leader_election.cc:302] T 00000000000000000000000000000000 P 4a061c25187f4add91c221ec71a9d76c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4a061c25187f4add91c221ec71a9d76c; no voters:
I20260809 12:22:07.383163 18924 leader_election.cc:288] T 00000000000000000000000000000000 P 4a061c25187f4add91c221ec71a9d76c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:22:07.383339 18938 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a061c25187f4add91c221ec71a9d76c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:22:07.384743 18938 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a061c25187f4add91c221ec71a9d76c [term 1 LEADER]: Becoming Leader. State: Replica: 4a061c25187f4add91c221ec71a9d76c, State: Running, Role: LEADER
I20260809 12:22:07.385416 18938 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a061c25187f4add91c221ec71a9d76c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a061c25187f4add91c221ec71a9d76c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37941 } }
I20260809 12:22:07.385576 18924 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a061c25187f4add91c221ec71a9d76c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:22:07.389421 18942 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a061c25187f4add91c221ec71a9d76c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a061c25187f4add91c221ec71a9d76c. Latest consensus state: current_term: 1 leader_uuid: "4a061c25187f4add91c221ec71a9d76c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a061c25187f4add91c221ec71a9d76c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37941 } } }
I20260809 12:22:07.390252 18942 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a061c25187f4add91c221ec71a9d76c [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:07.389933 18939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a061c25187f4add91c221ec71a9d76c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4a061c25187f4add91c221ec71a9d76c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4a061c25187f4add91c221ec71a9d76c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37941 } } }
I20260809 12:22:07.390726 18939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a061c25187f4add91c221ec71a9d76c [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:07.401117 18929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:07.381315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:46375 (local address 127.0.0.1:37941)
0809 12:22:07.381499 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:07.381516 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:07.381968 (+ 452us) server_negotiation.cc:407] Connection header received
0809 12:22:07.382095 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:07.382100 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:07.382193 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:07.382305 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:07.384351 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:07.385657 (+ 1306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:07.387028 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:07.387812 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:07.387948 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:07.391758 (+ 3810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:07.391768 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:07.391776 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:07.391828 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:07.396521 (+ 4693us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:07.397788 (+ 1267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:07.397797 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:07.397809 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:07.398030 (+ 221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:07.398818 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:07.398826 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:07.398832 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:07.399302 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:07.399611 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:07.400365 (+ 754us) server_negotiation.cc:299] Negotiation successful
0809 12:22:07.400592 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":59}
I20260809 12:22:07.402496 18950 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:22:07.407172 18950 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:22:07.415939 18950 catalog_manager.cc:1347] Generated new cluster ID: 2b78de4012c84c3eaaee358f2a01e0bd
I20260809 12:22:07.416069 18950 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:22:07.429054 18950 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:22:07.430279 18950 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:22:07.443682 18950 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a061c25187f4add91c221ec71a9d76c: Generated new TSK 0
I20260809 12:22:07.444742 18950 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:22:07.460711 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 18829
I20260809 12:22:07.461831 18919 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:22:07.554250 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 18829
W20260809 12:22:07.579723 18810 connection.cc:535] client connection to 127.0.0.1:37941 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37941): Connection reset by peer (error 104)
2026-08-09T12:22:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (2313 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:22:07 dist-test-slave-39l8 krb5kdc[18972](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:22:07 dist-test-slave-39l8 krb5kdc[18972](info): set up 2 sockets
Aug 09 12:22:07 dist-test-slave-39l8 krb5kdc[18972](info): commencing operation
krb5kdc: starting...
W20260809 12:22:08.855661 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.257s user 0.006s sys 0.043s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:22:09 dist-test-slave-39l8 krb5kdc[18972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278129, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:22:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:22:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:22:09.189529 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42131
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42131
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:22:09.534564 18991 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:22:09.534871 18991 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:22:09.534966 18991 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:22:09.541930 18991 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:22:09.542033 18991 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:22:09.542086 18991 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:22:09.542150 18991 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:22:09.542187 18991 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:22:09.551770 18991 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42131
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:22:09.553114 18991 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:22:09.554399 18991 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:22:09.566272 19006 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:09.566331 19007 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:09.566357 19010 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:22:09.567266 18991 server_base.cc:1034] running on GCE node
I20260809 12:22:09.568066 18991 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:22:09.569366 18991 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:22:09.570571 18991 hybrid_clock.cc:648] HybridClock initialized: now 1786278129570519 us; error 44 us; skew 500 ppm
I20260809 12:22:09.570945 18991 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:22:09 dist-test-slave-39l8 krb5kdc[18972](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278129, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:22:09.573932 18991 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:22:09.575531 18991 webserver.cc:460] Webserver started at http://127.0.0.1:37701/ using document root <none> and password file <none>
I20260809 12:22:09.576208 18991 fs_manager.cc:359] Metadata directory not provided
I20260809 12:22:09.576359 18991 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:22:09.576674 18991 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:22:09.579694 18991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "efd3bcd574de46608c92748df51fada9"
format_stamp: "Formatted at 2026-08-09 12:22:09 on dist-test-slave-39l8"
server_key: "c75d614f2bda7670513eb640a9848691"
server_key_iv: "8ab7e47027f569bbb8e94c6a0e1fb299"
server_key_version: "encryptionkey@0"
I20260809 12:22:09.580533 18991 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "efd3bcd574de46608c92748df51fada9"
format_stamp: "Formatted at 2026-08-09 12:22:09 on dist-test-slave-39l8"
server_key: "c75d614f2bda7670513eb640a9848691"
server_key_iv: "8ab7e47027f569bbb8e94c6a0e1fb299"
server_key_version: "encryptionkey@0"
I20260809 12:22:09.585739 18991 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260809 12:22:09.589682 19021 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:22:09.591081 18991 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260809 12:22:09.591279 18991 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "efd3bcd574de46608c92748df51fada9"
format_stamp: "Formatted at 2026-08-09 12:22:09 on dist-test-slave-39l8"
server_key: "c75d614f2bda7670513eb640a9848691"
server_key_iv: "8ab7e47027f569bbb8e94c6a0e1fb299"
server_key_version: "encryptionkey@0"
I20260809 12:22:09.591487 18991 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:22:09.604071 18991 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:22:09.608073 18991 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:22:09.608400 18991 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:22:09.626749 18991 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42131
I20260809 12:22:09.626829 19091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42131 every 8 connection(s)
I20260809 12:22:09.628135 18991 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:22:09.631749 19092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:22:09.633623 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 18991
I20260809 12:22:09.633824 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:22:09.634374 20191 external_mini_cluster.cc:1428] Setting key ed774b6501f05c5a7b149c6a83aeacbb
I20260809 12:22:09.644196 19092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efd3bcd574de46608c92748df51fada9: Bootstrap starting.
I20260809 12:22:09.647370 19099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:09.636156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:43375 (local address 127.0.0.1:42131)
0809 12:22:09.636918 (+ 762us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:09.636930 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:09.636970 (+ 40us) server_negotiation.cc:407] Connection header received
0809 12:22:09.638504 (+ 1534us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:09.638535 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:09.638970 (+ 435us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:09.639326 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:09.639848 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:09.641026 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:09.642269 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:09.642986 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:09.643179 (+ 193us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:09.644578 (+ 1399us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:09.644591 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:09.644607 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:22:09.645293 (+ 686us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:09.645388 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:22:09.645422 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:09.645584 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:09.645860 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 12:22:09.646043 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":534,"thread_start_us":337,"threads_started":1}
I20260809 12:22:09.648203 19092 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P efd3bcd574de46608c92748df51fada9: Neither blocks nor log segments found. Creating new log.
I20260809 12:22:09.649549 19092 log.cc:824] T 00000000000000000000000000000000 P efd3bcd574de46608c92748df51fada9: Log is configured to *not* fsync() on all Append() calls
I20260809 12:22:09.651563 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:22:09.652575 19092 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P efd3bcd574de46608c92748df51fada9: No bootstrap required, opened a new log
I20260809 12:22:09.660493 19092 raft_consensus.cc:348] T 00000000000000000000000000000000 P efd3bcd574de46608c92748df51fada9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd3bcd574de46608c92748df51fada9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42131 } }
I20260809 12:22:09.660946 19092 raft_consensus.cc:374] T 00000000000000000000000000000000 P efd3bcd574de46608c92748df51fada9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:22:09.661062 19092 raft_consensus.cc:729] T 00000000000000000000000000000000 P efd3bcd574de46608c92748df51fada9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: efd3bcd574de46608c92748df51fada9, State: Initialized, Role: FOLLOWER
I20260809 12:22:09.661758 19092 consensus_queue.cc:260] T 00000000000000000000000000000000 P efd3bcd574de46608c92748df51fada9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd3bcd574de46608c92748df51fada9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42131 } }
I20260809 12:22:09.662061 19092 raft_consensus.cc:388] T 00000000000000000000000000000000 P efd3bcd574de46608c92748df51fada9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:22:09.662190 19099 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:09.655830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:47553 (local address 127.0.0.1:42131)
0809 12:22:09.655983 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:09.655987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:09.656468 (+ 481us) server_negotiation.cc:407] Connection header received
0809 12:22:09.656694 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:09.656700 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:09.656764 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:09.656878 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:09.657433 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:09.658191 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:09.659541 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:09.659967 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:09.660097 (+ 130us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:09.661377 (+ 1280us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:09.661384 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:09.661388 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:22:09.661406 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:09.661473 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:22:09.661493 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:09.661636 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:09.661827 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 12:22:09.661875 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 12:22:09.662217 19092 raft_consensus.cc:482] T 00000000000000000000000000000000 P efd3bcd574de46608c92748df51fada9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:22:09.662434 19092 raft_consensus.cc:3037] T 00000000000000000000000000000000 P efd3bcd574de46608c92748df51fada9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:22:09.664887 19092 raft_consensus.cc:504] T 00000000000000000000000000000000 P efd3bcd574de46608c92748df51fada9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd3bcd574de46608c92748df51fada9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42131 } }
I20260809 12:22:09.665341 19092 leader_election.cc:302] T 00000000000000000000000000000000 P efd3bcd574de46608c92748df51fada9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: efd3bcd574de46608c92748df51fada9; no voters:
I20260809 12:22:09.665853 19092 leader_election.cc:288] T 00000000000000000000000000000000 P efd3bcd574de46608c92748df51fada9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:22:09.665992 19108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P efd3bcd574de46608c92748df51fada9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:22:09.666721 19108 raft_consensus.cc:686] T 00000000000000000000000000000000 P efd3bcd574de46608c92748df51fada9 [term 1 LEADER]: Becoming Leader. State: Replica: efd3bcd574de46608c92748df51fada9, State: Running, Role: LEADER
I20260809 12:22:09.667223 19108 consensus_queue.cc:237] T 00000000000000000000000000000000 P efd3bcd574de46608c92748df51fada9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd3bcd574de46608c92748df51fada9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42131 } }
I20260809 12:22:09.668021 19092 sys_catalog.cc:564] T 00000000000000000000000000000000 P efd3bcd574de46608c92748df51fada9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:22:09.670190 19109 sys_catalog.cc:455] T 00000000000000000000000000000000 P efd3bcd574de46608c92748df51fada9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "efd3bcd574de46608c92748df51fada9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd3bcd574de46608c92748df51fada9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42131 } } }
I20260809 12:22:09.670179 19110 sys_catalog.cc:455] T 00000000000000000000000000000000 P efd3bcd574de46608c92748df51fada9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader efd3bcd574de46608c92748df51fada9. Latest consensus state: current_term: 1 leader_uuid: "efd3bcd574de46608c92748df51fada9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "efd3bcd574de46608c92748df51fada9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42131 } } }
I20260809 12:22:09.670691 19110 sys_catalog.cc:458] T 00000000000000000000000000000000 P efd3bcd574de46608c92748df51fada9 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:09.671478 19109 sys_catalog.cc:458] T 00000000000000000000000000000000 P efd3bcd574de46608c92748df51fada9 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:09.672286 19113 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:22:09.677517 19113 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:22:09.687026 19113 catalog_manager.cc:1347] Generated new cluster ID: 3c13d16a15154b6db6d746faeae451b0
I20260809 12:22:09.687142 19113 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:22:09.695473 19113 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:22:09.696834 19113 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:22:09.703277 19113 catalog_manager.cc:5868] T 00000000000000000000000000000000 P efd3bcd574de46608c92748df51fada9: Generated new TSK 0
I20260809 12:22:09.704304 19113 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:22:09.721197 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 18991
W20260809 12:22:09.722672 19082 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.38:43375
W20260809 12:22:09.723598 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 18991: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:22:09.723798 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 18991
W20260809 12:22:09.752727 18967 connection.cc:535] client connection to 127.0.0.1:42131 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42131): Connection reset by peer (error 104)
2026-08-09T12:22:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (2172 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:22:09 dist-test-slave-39l8 krb5kdc[19137](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:22:09 dist-test-slave-39l8 krb5kdc[19137](info): set up 2 sockets
Aug 09 12:22:09 dist-test-slave-39l8 krb5kdc[19137](info): commencing operation
krb5kdc: starting...
W20260809 12:22:11.034978 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.263s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:22:11 dist-test-slave-39l8 krb5kdc[19137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278131, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:22:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:22:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:22:11.357810 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37523
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36437
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37523
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:22:11.692564 19158 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:22:11.692822 19158 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:22:11.692911 19158 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:22:11.700153 19158 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:22:11.700259 19158 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:22:11.700306 19158 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:22:11.700368 19158 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:22:11.700412 19158 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:22:11.709919 19158 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36437
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37523
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37523
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:22:11.711222 19158 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:22:11.712476 19158 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:22:11.725011 19166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:11.724888 19167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:11.724943 19170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:22:11.726150 19158 server_base.cc:1034] running on GCE node
I20260809 12:22:11.727010 19158 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:22:11.728427 19158 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:22:11.729669 19158 hybrid_clock.cc:648] HybridClock initialized: now 1786278131729615 us; error 68 us; skew 500 ppm
I20260809 12:22:11.730018 19158 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:22:11 dist-test-slave-39l8 krb5kdc[19137](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278131, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:22:11.736210 19158 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:22:11.737634 19158 webserver.cc:460] Webserver started at http://127.0.0.1:33581/ using document root <none> and password file <none>
I20260809 12:22:11.738238 19158 fs_manager.cc:359] Metadata directory not provided
I20260809 12:22:11.738363 19158 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:22:11.738636 19158 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:22:11.741724 19158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "7d205b9a2c8c4af58fbae51253edca6f"
format_stamp: "Formatted at 2026-08-09 12:22:11 on dist-test-slave-39l8"
server_key: "841365ebb061b0f70ed0e1191360b2a7"
server_key_iv: "31487af3349ae90697ee96107eff9950"
server_key_version: "encryptionkey@0"
I20260809 12:22:11.742422 19158 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "7d205b9a2c8c4af58fbae51253edca6f"
format_stamp: "Formatted at 2026-08-09 12:22:11 on dist-test-slave-39l8"
server_key: "841365ebb061b0f70ed0e1191360b2a7"
server_key_iv: "31487af3349ae90697ee96107eff9950"
server_key_version: "encryptionkey@0"
I20260809 12:22:11.747586 19158 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 12:22:11.750972 19179 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:22:11.752318 19158 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260809 12:22:11.752504 19158 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "7d205b9a2c8c4af58fbae51253edca6f"
format_stamp: "Formatted at 2026-08-09 12:22:11 on dist-test-slave-39l8"
server_key: "841365ebb061b0f70ed0e1191360b2a7"
server_key_iv: "31487af3349ae90697ee96107eff9950"
server_key_version: "encryptionkey@0"
I20260809 12:22:11.752718 19158 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:22:11.758332 19158 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:22:11.762583 19158 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:22:11.763022 19158 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:22:11.781750 19158 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37523
I20260809 12:22:11.781800 19245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37523 every 8 connection(s)
I20260809 12:22:11.783187 19158 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:22:11.785219 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 19158
I20260809 12:22:11.785414 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:22:11.785888 20191 external_mini_cluster.cc:1428] Setting key ae394fc19a4b9add24facb33394a988d
I20260809 12:22:11.787400 19246 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:22:11.794679 19251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:11.787889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:50177 (local address 127.0.0.1:37523)
0809 12:22:11.788713 (+ 824us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:11.788747 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:11.788798 (+ 51us) server_negotiation.cc:407] Connection header received
0809 12:22:11.790837 (+ 2039us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:11.790866 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:11.791346 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:11.791778 (+ 432us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:11.792798 (+ 1020us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:11.792806 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:11.792822 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:22:11.792844 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:11.792932 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:22:11.792968 (+ 36us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:11.793178 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:11.793317 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 12:22:11.793516 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":563,"thread_start_us":348,"threads_started":1}
I20260809 12:22:11.798197 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:22:11.802345 19246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d205b9a2c8c4af58fbae51253edca6f: Bootstrap starting.
I20260809 12:22:11.806239 19251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:11.803570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:44313 (local address 127.0.0.1:37523)
0809 12:22:11.803795 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:11.803804 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:11.804294 (+ 490us) server_negotiation.cc:407] Connection header received
0809 12:22:11.804467 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:11.804470 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:11.804515 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:11.804590 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:11.805566 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:11.805571 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:11.805574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:22:11.805589 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:11.805635 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:22:11.805650 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:11.805742 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:11.805887 (+ 145us) server_negotiation.cc:299] Negotiation successful
0809 12:22:11.805933 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 12:22:11.806404 19246 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d205b9a2c8c4af58fbae51253edca6f: Neither blocks nor log segments found. Creating new log.
I20260809 12:22:11.807667 19246 log.cc:824] T 00000000000000000000000000000000 P 7d205b9a2c8c4af58fbae51253edca6f: Log is configured to *not* fsync() on all Append() calls
I20260809 12:22:11.810173 19246 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d205b9a2c8c4af58fbae51253edca6f: No bootstrap required, opened a new log
I20260809 12:22:11.817487 19246 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d205b9a2c8c4af58fbae51253edca6f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d205b9a2c8c4af58fbae51253edca6f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37523 } }
I20260809 12:22:11.817826 19246 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d205b9a2c8c4af58fbae51253edca6f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:22:11.817966 19246 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d205b9a2c8c4af58fbae51253edca6f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d205b9a2c8c4af58fbae51253edca6f, State: Initialized, Role: FOLLOWER
I20260809 12:22:11.818620 19246 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d205b9a2c8c4af58fbae51253edca6f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d205b9a2c8c4af58fbae51253edca6f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37523 } }
I20260809 12:22:11.818907 19246 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d205b9a2c8c4af58fbae51253edca6f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:22:11.819016 19246 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d205b9a2c8c4af58fbae51253edca6f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:22:11.819151 19246 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d205b9a2c8c4af58fbae51253edca6f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:22:11.821249 19246 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d205b9a2c8c4af58fbae51253edca6f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d205b9a2c8c4af58fbae51253edca6f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37523 } }
I20260809 12:22:11.821674 19246 leader_election.cc:302] T 00000000000000000000000000000000 P 7d205b9a2c8c4af58fbae51253edca6f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d205b9a2c8c4af58fbae51253edca6f; no voters:
I20260809 12:22:11.822209 19246 leader_election.cc:288] T 00000000000000000000000000000000 P 7d205b9a2c8c4af58fbae51253edca6f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:22:11.822353 19260 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d205b9a2c8c4af58fbae51253edca6f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:22:11.823056 19260 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d205b9a2c8c4af58fbae51253edca6f [term 1 LEADER]: Becoming Leader. State: Replica: 7d205b9a2c8c4af58fbae51253edca6f, State: Running, Role: LEADER
I20260809 12:22:11.823587 19260 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d205b9a2c8c4af58fbae51253edca6f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d205b9a2c8c4af58fbae51253edca6f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37523 } }
I20260809 12:22:11.824272 19246 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d205b9a2c8c4af58fbae51253edca6f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:22:11.828455 19261 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d205b9a2c8c4af58fbae51253edca6f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d205b9a2c8c4af58fbae51253edca6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d205b9a2c8c4af58fbae51253edca6f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37523 } } }
I20260809 12:22:11.828987 19261 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d205b9a2c8c4af58fbae51253edca6f [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:11.830814 19267 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:22:11.831293 19263 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d205b9a2c8c4af58fbae51253edca6f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d205b9a2c8c4af58fbae51253edca6f. Latest consensus state: current_term: 1 leader_uuid: "7d205b9a2c8c4af58fbae51253edca6f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d205b9a2c8c4af58fbae51253edca6f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37523 } } }
I20260809 12:22:11.831660 19263 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d205b9a2c8c4af58fbae51253edca6f [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:11.838033 19267 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:22:11.847617 19267 catalog_manager.cc:1347] Generated new cluster ID: bed60a46260e4e638f353a4de060f2ff
I20260809 12:22:11.847745 19267 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:22:11.858639 19267 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:22:11.859017 19267 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:22:11.866005 19267 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d205b9a2c8c4af58fbae51253edca6f: Generated new TSK 0
I20260809 12:22:11.866748 19267 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:22:11.877220 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 19158
W20260809 12:22:11.878324 19240 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.38:50177
W20260809 12:22:11.879047 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 19158: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:22:11.879207 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 19158
2026-08-09T12:22:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (2157 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:22:12 dist-test-slave-39l8 krb5kdc[19297](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:22:12 dist-test-slave-39l8 krb5kdc[19297](info): set up 2 sockets
Aug 09 12:22:12 dist-test-slave-39l8 krb5kdc[19297](info): commencing operation
krb5kdc: starting...
W20260809 12:22:13.199370 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.270s user 0.000s sys 0.050s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:22:13 dist-test-slave-39l8 krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:22:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:22:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:22:13.533427 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33203
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45297
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:33203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:22:13.875075 19317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:22:13.875394 19317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:22:13.875492 19317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:22:13.882483 19317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:22:13.882588 19317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:22:13.882635 19317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:22:13.882697 19317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:22:13.882756 19317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:22:13.892230 19317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45297
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:33203
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:33203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:22:13.893468 19317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:22:13.894753 19317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:22:13.906739 19330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:13.906790 19325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:13.907714 19324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:22:13.907920 19317 server_base.cc:1034] running on GCE node
I20260809 12:22:13.908849 19317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:22:13.910189 19317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:22:13.911422 19317 hybrid_clock.cc:648] HybridClock initialized: now 1786278133911373 us; error 41 us; skew 500 ppm
I20260809 12:22:13.911774 19317 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:22:13 dist-test-slave-39l8 krb5kdc[19297](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278133, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:22:13.915042 19317 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:22:13.916467 19317 webserver.cc:460] Webserver started at http://127.0.0.1:42679/ using document root <none> and password file <none>
I20260809 12:22:13.917161 19317 fs_manager.cc:359] Metadata directory not provided
I20260809 12:22:13.917299 19317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:22:13.917618 19317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:22:13.921026 19317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "50cb5e3c7cac43d7b2bcd4b3ad967f89"
format_stamp: "Formatted at 2026-08-09 12:22:13 on dist-test-slave-39l8"
server_key: "909dea06fc14abfff9fa44491b6d0f8c"
server_key_iv: "1824edac95a03fe07013326549c0917c"
server_key_version: "encryptionkey@0"
I20260809 12:22:13.921887 19317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "50cb5e3c7cac43d7b2bcd4b3ad967f89"
format_stamp: "Formatted at 2026-08-09 12:22:13 on dist-test-slave-39l8"
server_key: "909dea06fc14abfff9fa44491b6d0f8c"
server_key_iv: "1824edac95a03fe07013326549c0917c"
server_key_version: "encryptionkey@0"
I20260809 12:22:13.927320 19317 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 12:22:13.931286 19339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:22:13.932792 19317 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:22:13.932945 19317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "50cb5e3c7cac43d7b2bcd4b3ad967f89"
format_stamp: "Formatted at 2026-08-09 12:22:13 on dist-test-slave-39l8"
server_key: "909dea06fc14abfff9fa44491b6d0f8c"
server_key_iv: "1824edac95a03fe07013326549c0917c"
server_key_version: "encryptionkey@0"
I20260809 12:22:13.933128 19317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:22:13.945398 19317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:22:13.949173 19317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:22:13.949406 19317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:22:13.967691 19317 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:33203
I20260809 12:22:13.967715 19412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:33203 every 8 connection(s)
I20260809 12:22:13.969079 19317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:22:13.972790 19413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:22:13.975085 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 19317
I20260809 12:22:13.975281 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:22:13.975827 20191 external_mini_cluster.cc:1428] Setting key bab7c02cd63e81d5d3d06e63314725a6
I20260809 12:22:13.985194 19413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50cb5e3c7cac43d7b2bcd4b3ad967f89: Bootstrap starting.
I20260809 12:22:13.988996 19413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50cb5e3c7cac43d7b2bcd4b3ad967f89: Neither blocks nor log segments found. Creating new log.
I20260809 12:22:13.990262 19413 log.cc:824] T 00000000000000000000000000000000 P 50cb5e3c7cac43d7b2bcd4b3ad967f89: Log is configured to *not* fsync() on all Append() calls
Aug 09 12:22:13 dist-test-slave-39l8 krb5kdc[19297](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278133, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 12:22:13.993036 19413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50cb5e3c7cac43d7b2bcd4b3ad967f89: No bootstrap required, opened a new log
I20260809 12:22:13.999339 19418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:13.977696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:51059 (local address 127.0.0.1:33203)
0809 12:22:13.978528 (+ 832us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:13.978539 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:13.978584 (+ 45us) server_negotiation.cc:407] Connection header received
0809 12:22:13.980474 (+ 1890us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:13.980495 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:13.980868 (+ 373us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:13.981171 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:13.982987 (+ 1816us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:13.984252 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:13.985949 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:13.986602 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:13.986800 (+ 198us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:13.991949 (+ 5149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:13.991956 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:13.991969 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:13.992018 (+ 49us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:13.995102 (+ 3084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:13.995924 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:13.995931 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:13.995940 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:13.996112 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:13.996614 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:13.996626 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:13.996628 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:13.997136 (+ 508us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:13.997502 (+ 366us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:13.997998 (+ 496us) server_negotiation.cc:299] Negotiation successful
0809 12:22:13.998319 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":627,"thread_start_us":372,"threads_started":1}
I20260809 12:22:14.000689 19413 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50cb5e3c7cac43d7b2bcd4b3ad967f89 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50cb5e3c7cac43d7b2bcd4b3ad967f89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33203 } }
I20260809 12:22:14.001027 19413 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50cb5e3c7cac43d7b2bcd4b3ad967f89 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:22:14.001149 19413 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50cb5e3c7cac43d7b2bcd4b3ad967f89 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50cb5e3c7cac43d7b2bcd4b3ad967f89, State: Initialized, Role: FOLLOWER
I20260809 12:22:14.001754 19413 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50cb5e3c7cac43d7b2bcd4b3ad967f89 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50cb5e3c7cac43d7b2bcd4b3ad967f89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33203 } }
I20260809 12:22:14.002015 19413 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50cb5e3c7cac43d7b2bcd4b3ad967f89 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:22:14.002169 19413 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50cb5e3c7cac43d7b2bcd4b3ad967f89 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:22:14.002384 19413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50cb5e3c7cac43d7b2bcd4b3ad967f89 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:22:14.003302 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:22:14.004721 19413 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50cb5e3c7cac43d7b2bcd4b3ad967f89 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50cb5e3c7cac43d7b2bcd4b3ad967f89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33203 } }
I20260809 12:22:14.005187 19413 leader_election.cc:302] T 00000000000000000000000000000000 P 50cb5e3c7cac43d7b2bcd4b3ad967f89 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50cb5e3c7cac43d7b2bcd4b3ad967f89; no voters:
I20260809 12:22:14.005834 19413 leader_election.cc:288] T 00000000000000000000000000000000 P 50cb5e3c7cac43d7b2bcd4b3ad967f89 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:22:14.005939 19421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50cb5e3c7cac43d7b2bcd4b3ad967f89 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:22:14.010977 19421 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50cb5e3c7cac43d7b2bcd4b3ad967f89 [term 1 LEADER]: Becoming Leader. State: Replica: 50cb5e3c7cac43d7b2bcd4b3ad967f89, State: Running, Role: LEADER
I20260809 12:22:14.011785 19421 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50cb5e3c7cac43d7b2bcd4b3ad967f89 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50cb5e3c7cac43d7b2bcd4b3ad967f89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33203 } }
I20260809 12:22:14.014428 19413 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50cb5e3c7cac43d7b2bcd4b3ad967f89 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:22:14.023468 19426 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50cb5e3c7cac43d7b2bcd4b3ad967f89 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50cb5e3c7cac43d7b2bcd4b3ad967f89. Latest consensus state: current_term: 1 leader_uuid: "50cb5e3c7cac43d7b2bcd4b3ad967f89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50cb5e3c7cac43d7b2bcd4b3ad967f89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33203 } } }
I20260809 12:22:14.023718 19428 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50cb5e3c7cac43d7b2bcd4b3ad967f89 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50cb5e3c7cac43d7b2bcd4b3ad967f89" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50cb5e3c7cac43d7b2bcd4b3ad967f89" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 33203 } } }
I20260809 12:22:14.024085 19428 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50cb5e3c7cac43d7b2bcd4b3ad967f89 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:14.024034 19426 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50cb5e3c7cac43d7b2bcd4b3ad967f89 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:14.027372 19418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:14.007926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:33645 (local address 127.0.0.1:33203)
0809 12:22:14.008135 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:14.008144 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:14.008702 (+ 558us) server_negotiation.cc:407] Connection header received
0809 12:22:14.008904 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:14.008909 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:14.008987 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:14.009087 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:14.010549 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:14.011719 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:14.013878 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:14.014674 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:14.014810 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:14.018474 (+ 3664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:14.018481 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:14.018486 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:14.018525 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:14.022934 (+ 4409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:14.024005 (+ 1071us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:14.024010 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:14.024013 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:14.024184 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:14.024909 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:14.024917 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:14.024923 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:14.025493 (+ 570us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:14.025823 (+ 330us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:14.026987 (+ 1164us) server_negotiation.cc:299] Negotiation successful
0809 12:22:14.027123 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 12:22:14.032027 19439 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:22:14.038072 19439 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:22:14.046914 19439 catalog_manager.cc:1347] Generated new cluster ID: 4746046d8c334757a49f39724d5c9e4a
I20260809 12:22:14.047026 19439 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:22:14.069537 19439 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:22:14.071588 19439 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:22:14.079430 19439 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50cb5e3c7cac43d7b2bcd4b3ad967f89: Generated new TSK 0
I20260809 12:22:14.080421 19439 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:22:14.085685 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 19317
I20260809 12:22:14.086864 19408 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:22:14.176045 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 19317
W20260809 12:22:14.203508 19295 connection.cc:535] client connection to 127.0.0.1:33203 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:33203): Connection reset by peer (error 104)
2026-08-09T12:22:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (2292 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:22:14 dist-test-slave-39l8 krb5kdc[19454](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:22:14 dist-test-slave-39l8 krb5kdc[19454](info): set up 2 sockets
Aug 09 12:22:14 dist-test-slave-39l8 krb5kdc[19454](info): commencing operation
krb5kdc: starting...
W20260809 12:22:15.484469 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.262s user 0.003s sys 0.045s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:22:15 dist-test-slave-39l8 krb5kdc[19454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278135, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:22:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:22:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:22:15.805979 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41621
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41621
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:22:16.149283 19477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:22:16.149648 19477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:22:16.149816 19477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:22:16.156978 19477 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:22:16.157131 19477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:22:16.157249 19477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:22:16.157364 19477 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:22:16.157454 19477 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:22:16.167261 19477 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41621
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41621
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:22:16.168579 19477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:22:16.170019 19477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:22:16.182173 19485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:16.182209 19488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:16.182088 19486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:22:16.182825 19477 server_base.cc:1034] running on GCE node
I20260809 12:22:16.183687 19477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:22:16.185170 19477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:22:16.186404 19477 hybrid_clock.cc:648] HybridClock initialized: now 1786278136186361 us; error 56 us; skew 500 ppm
I20260809 12:22:16.186822 19477 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:22:16 dist-test-slave-39l8 krb5kdc[19454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278136, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:22:16.189929 19477 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:22:16.191344 19477 webserver.cc:460] Webserver started at http://127.0.0.1:37467/ using document root <none> and password file <none>
I20260809 12:22:16.192010 19477 fs_manager.cc:359] Metadata directory not provided
I20260809 12:22:16.192116 19477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:22:16.192380 19477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:22:16.195415 19477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "6fe3cc9bc72a4f1badb9e83f3364e7be"
format_stamp: "Formatted at 2026-08-09 12:22:16 on dist-test-slave-39l8"
server_key: "61d410d6314b963cd84178b7e2a266f9"
server_key_iv: "8987ebc78572e76ddb5d42984161a4b3"
server_key_version: "encryptionkey@0"
I20260809 12:22:16.196158 19477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "6fe3cc9bc72a4f1badb9e83f3364e7be"
format_stamp: "Formatted at 2026-08-09 12:22:16 on dist-test-slave-39l8"
server_key: "61d410d6314b963cd84178b7e2a266f9"
server_key_iv: "8987ebc78572e76ddb5d42984161a4b3"
server_key_version: "encryptionkey@0"
I20260809 12:22:16.201268 19477 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:22:16.204677 19496 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:22:16.206035 19477 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:22:16.206225 19477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "6fe3cc9bc72a4f1badb9e83f3364e7be"
format_stamp: "Formatted at 2026-08-09 12:22:16 on dist-test-slave-39l8"
server_key: "61d410d6314b963cd84178b7e2a266f9"
server_key_iv: "8987ebc78572e76ddb5d42984161a4b3"
server_key_version: "encryptionkey@0"
I20260809 12:22:16.206421 19477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:22:16.233757 19477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:22:16.238005 19477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:22:16.238313 19477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:22:16.257125 19477 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41621
I20260809 12:22:16.257164 19570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41621 every 8 connection(s)
I20260809 12:22:16.258510 19477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:22:16.262179 19571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:22:16.266198 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 19477
I20260809 12:22:16.266383 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:22:16.266928 20191 external_mini_cluster.cc:1428] Setting key 4bfe3afc1b61bc16f26b529dc8884cd3
I20260809 12:22:16.273072 19571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fe3cc9bc72a4f1badb9e83f3364e7be: Bootstrap starting.
I20260809 12:22:16.276575 19571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6fe3cc9bc72a4f1badb9e83f3364e7be: Neither blocks nor log segments found. Creating new log.
I20260809 12:22:16.277962 19571 log.cc:824] T 00000000000000000000000000000000 P 6fe3cc9bc72a4f1badb9e83f3364e7be: Log is configured to *not* fsync() on all Append() calls
I20260809 12:22:16.280799 19575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:16.268632 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:53745 (local address 127.0.0.1:41621)
0809 12:22:16.269364 (+ 732us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:16.269375 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:16.269415 (+ 40us) server_negotiation.cc:407] Connection header received
0809 12:22:16.271574 (+ 2159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:16.271596 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:16.271948 (+ 352us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:16.272294 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:16.272947 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:16.274351 (+ 1404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:16.276200 (+ 1849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:16.276918 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:16.277088 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:16.278330 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:16.278337 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:16.278350 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:22:16.279154 (+ 804us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:16.279248 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:22:16.279283 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:16.279432 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:16.279744 (+ 312us) server_negotiation.cc:299] Negotiation successful
0809 12:22:16.279915 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":531,"thread_start_us":350,"threads_started":1}
I20260809 12:22:16.280946 19571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6fe3cc9bc72a4f1badb9e83f3364e7be: No bootstrap required, opened a new log
I20260809 12:22:16.284603 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:22:16.290123 19571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6fe3cc9bc72a4f1badb9e83f3364e7be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fe3cc9bc72a4f1badb9e83f3364e7be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41621 } }
I20260809 12:22:16.290504 19571 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6fe3cc9bc72a4f1badb9e83f3364e7be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:22:16.290628 19571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6fe3cc9bc72a4f1badb9e83f3364e7be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6fe3cc9bc72a4f1badb9e83f3364e7be, State: Initialized, Role: FOLLOWER
I20260809 12:22:16.291214 19571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6fe3cc9bc72a4f1badb9e83f3364e7be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fe3cc9bc72a4f1badb9e83f3364e7be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41621 } }
I20260809 12:22:16.291518 19571 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6fe3cc9bc72a4f1badb9e83f3364e7be [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:22:16.291674 19571 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6fe3cc9bc72a4f1badb9e83f3364e7be [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:22:16.291864 19571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6fe3cc9bc72a4f1badb9e83f3364e7be [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:22:16.294478 19571 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6fe3cc9bc72a4f1badb9e83f3364e7be [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fe3cc9bc72a4f1badb9e83f3364e7be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41621 } }
I20260809 12:22:16.295106 19571 leader_election.cc:302] T 00000000000000000000000000000000 P 6fe3cc9bc72a4f1badb9e83f3364e7be [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6fe3cc9bc72a4f1badb9e83f3364e7be; no voters:
I20260809 12:22:16.295737 19571 leader_election.cc:288] T 00000000000000000000000000000000 P 6fe3cc9bc72a4f1badb9e83f3364e7be [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:22:16.295807 19584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6fe3cc9bc72a4f1badb9e83f3364e7be [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:22:16.297348 19575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:16.289806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:60599 (local address 127.0.0.1:41621)
0809 12:22:16.289989 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:16.289995 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:16.290375 (+ 380us) server_negotiation.cc:407] Connection header received
0809 12:22:16.290596 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:16.290602 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:16.290687 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:16.290816 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:16.291401 (+ 585us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:16.292552 (+ 1151us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:16.293794 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:16.294519 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:16.294654 (+ 135us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:16.296034 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:16.296043 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:16.296050 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:22:16.296080 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:16.296166 (+ 86us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:22:16.296196 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:16.296875 (+ 679us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:16.296953 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 12:22:16.297001 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 12:22:16.298022 19571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6fe3cc9bc72a4f1badb9e83f3364e7be [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:22:16.298189 19584 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6fe3cc9bc72a4f1badb9e83f3364e7be [term 1 LEADER]: Becoming Leader. State: Replica: 6fe3cc9bc72a4f1badb9e83f3364e7be, State: Running, Role: LEADER
I20260809 12:22:16.298898 19584 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6fe3cc9bc72a4f1badb9e83f3364e7be [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fe3cc9bc72a4f1badb9e83f3364e7be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41621 } }
I20260809 12:22:16.302196 19585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fe3cc9bc72a4f1badb9e83f3364e7be [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6fe3cc9bc72a4f1badb9e83f3364e7be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fe3cc9bc72a4f1badb9e83f3364e7be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41621 } } }
I20260809 12:22:16.302678 19585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fe3cc9bc72a4f1badb9e83f3364e7be [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:16.302174 19587 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6fe3cc9bc72a4f1badb9e83f3364e7be [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6fe3cc9bc72a4f1badb9e83f3364e7be. Latest consensus state: current_term: 1 leader_uuid: "6fe3cc9bc72a4f1badb9e83f3364e7be" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6fe3cc9bc72a4f1badb9e83f3364e7be" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41621 } } }
I20260809 12:22:16.302986 19587 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6fe3cc9bc72a4f1badb9e83f3364e7be [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:16.304277 19601 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:22:16.309293 19601 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:22:16.321233 19601 catalog_manager.cc:1347] Generated new cluster ID: 87917307f64a4ae4a1b99248bf3b527d
I20260809 12:22:16.321373 19601 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:22:16.344975 19601 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:22:16.346844 19601 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:22:16.356014 19601 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6fe3cc9bc72a4f1badb9e83f3364e7be: Generated new TSK 0
I20260809 12:22:16.356994 19601 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:22:16.373533 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 19477
W20260809 12:22:16.375011 19565 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.38:53745
W20260809 12:22:16.375801 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 19477: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:22:16.375949 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 19477
W20260809 12:22:16.398303 19452 connection.cc:535] client connection to 127.0.0.1:41621 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41621): Connection reset by peer (error 104)
2026-08-09T12:22:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (2194 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:22:16 dist-test-slave-39l8 krb5kdc[19621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:22:16 dist-test-slave-39l8 krb5kdc[19621](info): set up 2 sockets
Aug 09 12:22:16 dist-test-slave-39l8 krb5kdc[19621](info): commencing operation
krb5kdc: starting...
W20260809 12:22:17.680853 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.263s user 0.000s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:22:17 dist-test-slave-39l8 krb5kdc[19621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278137, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:22:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:22:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 12:22:18.009590 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45875
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45875
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:22:18.362106 19638 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:22:18.362394 19638 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:22:18.362571 19638 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:22:18.369621 19638 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:22:18.369762 19638 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:22:18.369858 19638 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:22:18.369978 19638 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:22:18.370077 19638 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:22:18.379797 19638 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45875
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45875
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:22:18.381070 19638 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:22:18.382396 19638 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:22:18.395378 19646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:18.395313 19647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:18.395351 19650 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:22:18.395644 19638 server_base.cc:1034] running on GCE node
I20260809 12:22:18.396627 19638 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:22:18.398000 19638 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:22:18.399227 19638 hybrid_clock.cc:648] HybridClock initialized: now 1786278138399189 us; error 41 us; skew 500 ppm
I20260809 12:22:18.399588 19638 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:22:18 dist-test-slave-39l8 krb5kdc[19621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278138, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:22:18.408823 19638 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 12:22:18.410276 19638 webserver.cc:460] Webserver started at http://127.0.0.1:35071/ using document root <none> and password file <none>
I20260809 12:22:18.411085 19638 fs_manager.cc:359] Metadata directory not provided
I20260809 12:22:18.411243 19638 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:22:18.411571 19638 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:22:18.414474 19638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "d2ae81b74b0348ee9087b1a59157b166"
format_stamp: "Formatted at 2026-08-09 12:22:18 on dist-test-slave-39l8"
server_key: "c7e9f4297e61eeb7a4d4a1756d2aead9"
server_key_iv: "a6f545f2423094226dba370354329130"
server_key_version: "encryptionkey@0"
I20260809 12:22:18.415316 19638 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "d2ae81b74b0348ee9087b1a59157b166"
format_stamp: "Formatted at 2026-08-09 12:22:18 on dist-test-slave-39l8"
server_key: "c7e9f4297e61eeb7a4d4a1756d2aead9"
server_key_iv: "a6f545f2423094226dba370354329130"
server_key_version: "encryptionkey@0"
I20260809 12:22:18.420956 19638 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:22:18.424408 19663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:22:18.425711 19638 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:22:18.425890 19638 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "d2ae81b74b0348ee9087b1a59157b166"
format_stamp: "Formatted at 2026-08-09 12:22:18 on dist-test-slave-39l8"
server_key: "c7e9f4297e61eeb7a4d4a1756d2aead9"
server_key_iv: "a6f545f2423094226dba370354329130"
server_key_version: "encryptionkey@0"
I20260809 12:22:18.426085 19638 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:22:18.431907 19638 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:22:18.435753 19638 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:22:18.436053 19638 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:22:18.454519 19638 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45875
I20260809 12:22:18.454568 19745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45875 every 8 connection(s)
I20260809 12:22:18.455924 19638 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:22:18.459641 19746 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:22:18.461995 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 19638
I20260809 12:22:18.462203 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:22:18.462833 20191 external_mini_cluster.cc:1428] Setting key edc3de03544bc49d8efe8b5f4700c0f3
I20260809 12:22:18.470690 19749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:18.464699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:39113 (local address 127.0.0.1:45875)
0809 12:22:18.465546 (+ 847us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:18.465558 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:18.465600 (+ 42us) server_negotiation.cc:407] Connection header received
0809 12:22:18.467337 (+ 1737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:18.467360 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:18.467688 (+ 328us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:18.467998 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:18.469081 (+ 1083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:18.469087 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:18.469099 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:22:18.469112 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:18.469176 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:22:18.469194 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:18.469373 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:18.469471 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 12:22:18.469697 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":423,"threads_started":1}
I20260809 12:22:18.472594 19746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2ae81b74b0348ee9087b1a59157b166: Bootstrap starting.
I20260809 12:22:18.474579 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:22:18.475996 19746 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d2ae81b74b0348ee9087b1a59157b166: Neither blocks nor log segments found. Creating new log.
I20260809 12:22:18.477017 19746 log.cc:824] T 00000000000000000000000000000000 P d2ae81b74b0348ee9087b1a59157b166: Log is configured to *not* fsync() on all Append() calls
I20260809 12:22:18.480719 19746 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d2ae81b74b0348ee9087b1a59157b166: No bootstrap required, opened a new log
I20260809 12:22:18.482167 19749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:18.479260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.38:37953 (local address 127.0.0.1:45875)
0809 12:22:18.479543 (+ 283us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:18.479550 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:18.480296 (+ 746us) server_negotiation.cc:407] Connection header received
0809 12:22:18.480448 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:18.480453 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:18.480512 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:18.480630 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:18.481542 (+ 912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:18.481548 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:18.481552 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:22:18.481569 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:18.481620 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:22:18.481639 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:18.481750 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:18.481835 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 12:22:18.481894 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260809 12:22:18.488332 19746 raft_consensus.cc:348] T 00000000000000000000000000000000 P d2ae81b74b0348ee9087b1a59157b166 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ae81b74b0348ee9087b1a59157b166" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45875 } }
I20260809 12:22:18.488657 19746 raft_consensus.cc:374] T 00000000000000000000000000000000 P d2ae81b74b0348ee9087b1a59157b166 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:22:18.488729 19746 raft_consensus.cc:729] T 00000000000000000000000000000000 P d2ae81b74b0348ee9087b1a59157b166 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d2ae81b74b0348ee9087b1a59157b166, State: Initialized, Role: FOLLOWER
I20260809 12:22:18.489202 19746 consensus_queue.cc:260] T 00000000000000000000000000000000 P d2ae81b74b0348ee9087b1a59157b166 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ae81b74b0348ee9087b1a59157b166" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45875 } }
I20260809 12:22:18.489391 19746 raft_consensus.cc:388] T 00000000000000000000000000000000 P d2ae81b74b0348ee9087b1a59157b166 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:22:18.489487 19746 raft_consensus.cc:482] T 00000000000000000000000000000000 P d2ae81b74b0348ee9087b1a59157b166 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:22:18.489598 19746 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d2ae81b74b0348ee9087b1a59157b166 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:22:18.491794 19746 raft_consensus.cc:504] T 00000000000000000000000000000000 P d2ae81b74b0348ee9087b1a59157b166 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ae81b74b0348ee9087b1a59157b166" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45875 } }
I20260809 12:22:18.492336 19746 leader_election.cc:302] T 00000000000000000000000000000000 P d2ae81b74b0348ee9087b1a59157b166 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d2ae81b74b0348ee9087b1a59157b166; no voters:
I20260809 12:22:18.492910 19746 leader_election.cc:288] T 00000000000000000000000000000000 P d2ae81b74b0348ee9087b1a59157b166 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:22:18.493040 19760 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d2ae81b74b0348ee9087b1a59157b166 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:22:18.493786 19760 raft_consensus.cc:686] T 00000000000000000000000000000000 P d2ae81b74b0348ee9087b1a59157b166 [term 1 LEADER]: Becoming Leader. State: Replica: d2ae81b74b0348ee9087b1a59157b166, State: Running, Role: LEADER
I20260809 12:22:18.494369 19760 consensus_queue.cc:237] T 00000000000000000000000000000000 P d2ae81b74b0348ee9087b1a59157b166 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ae81b74b0348ee9087b1a59157b166" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45875 } }
I20260809 12:22:18.495012 19746 sys_catalog.cc:564] T 00000000000000000000000000000000 P d2ae81b74b0348ee9087b1a59157b166 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:22:18.497390 19765 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2ae81b74b0348ee9087b1a59157b166 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d2ae81b74b0348ee9087b1a59157b166. Latest consensus state: current_term: 1 leader_uuid: "d2ae81b74b0348ee9087b1a59157b166" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ae81b74b0348ee9087b1a59157b166" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45875 } } }
I20260809 12:22:18.497417 19764 sys_catalog.cc:455] T 00000000000000000000000000000000 P d2ae81b74b0348ee9087b1a59157b166 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d2ae81b74b0348ee9087b1a59157b166" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d2ae81b74b0348ee9087b1a59157b166" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45875 } } }
I20260809 12:22:18.498232 19765 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2ae81b74b0348ee9087b1a59157b166 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:18.498257 19764 sys_catalog.cc:458] T 00000000000000000000000000000000 P d2ae81b74b0348ee9087b1a59157b166 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:18.501103 19781 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:22:18.505617 19781 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:22:18.513973 19781 catalog_manager.cc:1347] Generated new cluster ID: b02570ec86e3469cb1416d500dac1349
I20260809 12:22:18.514089 19781 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:22:18.533946 19781 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:22:18.534328 19781 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:22:18.543319 19781 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d2ae81b74b0348ee9087b1a59157b166: Generated new TSK 0
I20260809 12:22:18.544216 19781 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:22:18.558535 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 19638
W20260809 12:22:18.559640 19737 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 10.240.0.38:39113
W20260809 12:22:18.560376 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 19638: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:22:18.560520 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 19638
2026-08-09T12:22:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (2189 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:22:18 dist-test-slave-39l8 krb5kdc[19795](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:22:18 dist-test-slave-39l8 krb5kdc[19795](info): set up 2 sockets
Aug 09 12:22:18 dist-test-slave-39l8 krb5kdc[19795](info): commencing operation
krb5kdc: starting...
W20260809 12:22:19.881701 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.274s user 0.004s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:22:20 dist-test-slave-39l8 krb5kdc[19795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:22:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:22:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:22:20.213543 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:40257
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:40619
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:40257
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:22:20.570354 19822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:22:20.570710 19822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:22:20.570899 19822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:22:20.578441 19822 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:22:20.578613 19822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:22:20.578752 19822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:22:20.578855 19822 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:22:20.578927 19822 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:22:20.589208 19822 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:40619
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:40257
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:40257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:22:20.590693 19822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:22:20.592176 19822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:22:20.604530 19828 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:20.604565 19829 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:20.604998 19832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:22:20.605507 19822 server_base.cc:1034] running on GCE node
I20260809 12:22:20.606418 19822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:22:20.607978 19822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:22:20.609202 19822 hybrid_clock.cc:648] HybridClock initialized: now 1786278140609167 us; error 33 us; skew 500 ppm
I20260809 12:22:20.609592 19822 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:22:20 dist-test-slave-39l8 krb5kdc[19795](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278140, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:22:20.612851 19822 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:22:20.614459 19822 webserver.cc:460] Webserver started at http://127.19.183.254:44803/ using document root <none> and password file <none>
I20260809 12:22:20.615265 19822 fs_manager.cc:359] Metadata directory not provided
I20260809 12:22:20.615423 19822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:22:20.615767 19822 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:22:20.618952 19822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "8901f6046a4b4bd0937be543f3bfa035"
format_stamp: "Formatted at 2026-08-09 12:22:20 on dist-test-slave-39l8"
server_key: "744330e99d71b002e678ba497b1204d7"
server_key_iv: "8118b828ae35a2991474df29c968707f"
server_key_version: "encryptionkey@0"
I20260809 12:22:20.619779 19822 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "8901f6046a4b4bd0937be543f3bfa035"
format_stamp: "Formatted at 2026-08-09 12:22:20 on dist-test-slave-39l8"
server_key: "744330e99d71b002e678ba497b1204d7"
server_key_iv: "8118b828ae35a2991474df29c968707f"
server_key_version: "encryptionkey@0"
I20260809 12:22:20.625181 19822 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260809 12:22:20.628871 19840 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:22:20.630196 19822 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:22:20.630414 19822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "8901f6046a4b4bd0937be543f3bfa035"
format_stamp: "Formatted at 2026-08-09 12:22:20 on dist-test-slave-39l8"
server_key: "744330e99d71b002e678ba497b1204d7"
server_key_iv: "8118b828ae35a2991474df29c968707f"
server_key_version: "encryptionkey@0"
I20260809 12:22:20.630650 19822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:22:20.645848 19822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:22:20.650070 19822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:22:20.650388 19822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:22:20.669955 19822 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:40257
I20260809 12:22:20.669974 19917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:40257 every 8 connection(s)
I20260809 12:22:20.671443 19822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:22:20.675379 19918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:22:20.676985 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 19822
I20260809 12:22:20.677224 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:22:20.677836 20191 external_mini_cluster.cc:1428] Setting key 5e691ac3b75b9a28cc52906351382efd
I20260809 12:22:20.690493 19918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8901f6046a4b4bd0937be543f3bfa035: Bootstrap starting.
Aug 09 12:22:20 dist-test-slave-39l8 krb5kdc[19795](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278140, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:22:20.694777 19918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8901f6046a4b4bd0937be543f3bfa035: Neither blocks nor log segments found. Creating new log.
I20260809 12:22:20.695987 19918 log.cc:824] T 00000000000000000000000000000000 P 8901f6046a4b4bd0937be543f3bfa035: Log is configured to *not* fsync() on all Append() calls
I20260809 12:22:20.699105 19918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8901f6046a4b4bd0937be543f3bfa035: No bootstrap required, opened a new log
I20260809 12:22:20.703212 19922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:20.680100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51683 (local address 127.19.183.254:40257)
0809 12:22:20.680979 (+ 879us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:20.680992 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:20.681045 (+ 53us) server_negotiation.cc:407] Connection header received
0809 12:22:20.682815 (+ 1770us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:20.682842 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:20.683287 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:20.683625 (+ 338us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:20.685565 (+ 1940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:20.687066 (+ 1501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:20.688994 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:20.689682 (+ 688us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:20.689819 (+ 137us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:20.695227 (+ 5408us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:20.695238 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:20.695259 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:20.695310 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:20.698539 (+ 3229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:20.699509 (+ 970us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:20.699516 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:20.699523 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:20.699710 (+ 187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:20.700230 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:20.700233 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:20.700235 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:20.700802 (+ 567us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:20.701193 (+ 391us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:20.701737 (+ 544us) server_negotiation.cc:299] Negotiation successful
0809 12:22:20.702063 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":634,"thread_start_us":428,"threads_started":1}
I20260809 12:22:20.707463 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:22:20.708096 19918 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8901f6046a4b4bd0937be543f3bfa035 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8901f6046a4b4bd0937be543f3bfa035" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40257 } }
I20260809 12:22:20.708501 19918 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8901f6046a4b4bd0937be543f3bfa035 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:22:20.708609 19918 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8901f6046a4b4bd0937be543f3bfa035 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8901f6046a4b4bd0937be543f3bfa035, State: Initialized, Role: FOLLOWER
I20260809 12:22:20.709237 19918 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8901f6046a4b4bd0937be543f3bfa035 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8901f6046a4b4bd0937be543f3bfa035" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40257 } }
I20260809 12:22:20.709515 19918 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8901f6046a4b4bd0937be543f3bfa035 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:22:20.709676 19918 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8901f6046a4b4bd0937be543f3bfa035 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:22:20.709851 19918 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8901f6046a4b4bd0937be543f3bfa035 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:22:20.712968 19918 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8901f6046a4b4bd0937be543f3bfa035 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8901f6046a4b4bd0937be543f3bfa035" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40257 } }
I20260809 12:22:20.713666 19918 leader_election.cc:302] T 00000000000000000000000000000000 P 8901f6046a4b4bd0937be543f3bfa035 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8901f6046a4b4bd0937be543f3bfa035; no voters:
I20260809 12:22:20.714377 19918 leader_election.cc:288] T 00000000000000000000000000000000 P 8901f6046a4b4bd0937be543f3bfa035 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:22:20.714502 19932 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8901f6046a4b4bd0937be543f3bfa035 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:22:20.716900 19918 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8901f6046a4b4bd0937be543f3bfa035 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:22:20.721210 19932 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8901f6046a4b4bd0937be543f3bfa035 [term 1 LEADER]: Becoming Leader. State: Replica: 8901f6046a4b4bd0937be543f3bfa035, State: Running, Role: LEADER
I20260809 12:22:20.721814 19932 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8901f6046a4b4bd0937be543f3bfa035 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8901f6046a4b4bd0937be543f3bfa035" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40257 } }
I20260809 12:22:20.725776 19933 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8901f6046a4b4bd0937be543f3bfa035 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8901f6046a4b4bd0937be543f3bfa035" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8901f6046a4b4bd0937be543f3bfa035" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40257 } } }
I20260809 12:22:20.726240 19933 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8901f6046a4b4bd0937be543f3bfa035 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:20.731055 19937 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8901f6046a4b4bd0937be543f3bfa035 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8901f6046a4b4bd0937be543f3bfa035. Latest consensus state: current_term: 1 leader_uuid: "8901f6046a4b4bd0937be543f3bfa035" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8901f6046a4b4bd0937be543f3bfa035" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40257 } } }
I20260809 12:22:20.732056 19937 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8901f6046a4b4bd0937be543f3bfa035 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:20.734761 19922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:20.712283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57569 (local address 127.19.183.254:40257)
0809 12:22:20.712491 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:20.712500 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:20.712850 (+ 350us) server_negotiation.cc:407] Connection header received
0809 12:22:20.713065 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:20.713071 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:20.713149 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:20.713257 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:20.715449 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:20.716632 (+ 1183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:20.718724 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:20.719691 (+ 967us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:20.719891 (+ 200us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:20.723794 (+ 3903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:20.723804 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:20.723812 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:20.723862 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:20.729053 (+ 5191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:20.730538 (+ 1485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:20.730547 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:20.730553 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:20.730787 (+ 234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:20.731480 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:20.731486 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:20.731492 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:20.732088 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:20.733722 (+ 1634us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:20.733955 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 12:22:20.734222 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 12:22:20.738358 19947 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:22:20.744146 19947 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:22:20.753368 19947 catalog_manager.cc:1347] Generated new cluster ID: 9cf7adb5d167442f85f03ffa88a0c4b5
I20260809 12:22:20.753499 19947 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:22:20.782913 19947 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:22:20.784341 19947 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:22:20.791622 19947 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8901f6046a4b4bd0937be543f3bfa035: Generated new TSK 0
I20260809 12:22:20.792562 19947 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:22:20.811333 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 19822
I20260809 12:22:20.812623 19910 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:22:20.909371 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 19822
W20260809 12:22:20.935814 19793 connection.cc:535] client connection to 127.19.183.254:40257 recv error: Network error: failed to read from TLS socket (remote: 127.19.183.254:40257): Connection reset by peer (error 104)
2026-08-09T12:22:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (2347 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:22:21 dist-test-slave-39l8 krb5kdc[19974](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:22:21 dist-test-slave-39l8 krb5kdc[19974](info): set up 2 sockets
Aug 09 12:22:21 dist-test-slave-39l8 krb5kdc[19974](info): commencing operation
krb5kdc: starting...
W20260809 12:22:22.270851 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.316s user 0.000s sys 0.059s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:22:22 dist-test-slave-39l8 krb5kdc[19974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278142, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:22:22Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:22:22Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:22:22.644050 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:36571
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:39865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:36571
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:22:23.023844 19990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:22:23.024307 19990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:22:23.024487 19990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:22:23.032728 19990 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:22:23.032913 19990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:22:23.033013 19990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:22:23.033129 19990 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:22:23.033215 19990 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:22:23.043641 19990 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:39865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:36571
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:36571
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:22:23.045120 19990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:22:23.046569 19990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:22:23.059377 20000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:23.059453 20005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:23.059458 20001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:22:23.060278 19990 server_base.cc:1034] running on GCE node
I20260809 12:22:23.061115 19990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:22:23.062647 19990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:22:23.063926 19990 hybrid_clock.cc:648] HybridClock initialized: now 1786278143063873 us; error 47 us; skew 500 ppm
I20260809 12:22:23.064312 19990 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:22:23 dist-test-slave-39l8 krb5kdc[19974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278143, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:22:23.067881 19990 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:22:23.069667 19990 webserver.cc:460] Webserver started at http://127.19.183.254:33055/ using document root <none> and password file <none>
I20260809 12:22:23.070426 19990 fs_manager.cc:359] Metadata directory not provided
I20260809 12:22:23.070582 19990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:22:23.070990 19990 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:22:23.074218 19990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "ee057a9504844b4ca8b53f8e4effc594"
format_stamp: "Formatted at 2026-08-09 12:22:23 on dist-test-slave-39l8"
server_key: "a55bddf9f3227ff5f0327c24e5cabaff"
server_key_iv: "f9e7635108d50c7982bdaacc84a0e22f"
server_key_version: "encryptionkey@0"
I20260809 12:22:23.075147 19990 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "ee057a9504844b4ca8b53f8e4effc594"
format_stamp: "Formatted at 2026-08-09 12:22:23 on dist-test-slave-39l8"
server_key: "a55bddf9f3227ff5f0327c24e5cabaff"
server_key_iv: "f9e7635108d50c7982bdaacc84a0e22f"
server_key_version: "encryptionkey@0"
I20260809 12:22:23.080778 19990 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 12:22:23.084663 20018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:22:23.086156 19990 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:22:23.086366 19990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "ee057a9504844b4ca8b53f8e4effc594"
format_stamp: "Formatted at 2026-08-09 12:22:23 on dist-test-slave-39l8"
server_key: "a55bddf9f3227ff5f0327c24e5cabaff"
server_key_iv: "f9e7635108d50c7982bdaacc84a0e22f"
server_key_version: "encryptionkey@0"
I20260809 12:22:23.086596 19990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:22:23.120792 19990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:22:23.125664 19990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:22:23.125986 19990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:22:23.146025 19990 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:36571
I20260809 12:22:23.146050 20100 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:36571 every 8 connection(s)
I20260809 12:22:23.147583 19990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:22:23.151670 20102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:22:23.153326 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 19990
I20260809 12:22:23.153543 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:22:23.154126 20191 external_mini_cluster.cc:1428] Setting key 8f71f7d3d90855dfda18560ecfe090d5
I20260809 12:22:23.165822 20102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee057a9504844b4ca8b53f8e4effc594: Bootstrap starting.
I20260809 12:22:23.167919 20107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:23.156145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57641 (local address 127.19.183.254:36571)
0809 12:22:23.157055 (+ 910us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:23.157071 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:23.157120 (+ 49us) server_negotiation.cc:407] Connection header received
0809 12:22:23.158818 (+ 1698us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:23.158847 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:23.159289 (+ 442us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:23.159646 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:23.160272 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:23.161550 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:23.163070 (+ 1520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:23.163830 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:23.164046 (+ 216us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:23.165446 (+ 1400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:23.165458 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:23.165476 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:22:23.166109 (+ 633us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:23.166203 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:22:23.166229 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:23.166410 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:23.166731 (+ 321us) server_negotiation.cc:299] Negotiation successful
0809 12:22:23.166913 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":682,"thread_start_us":431,"threads_started":1}
I20260809 12:22:23.169361 20102 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee057a9504844b4ca8b53f8e4effc594: Neither blocks nor log segments found. Creating new log.
I20260809 12:22:23.170563 20102 log.cc:824] T 00000000000000000000000000000000 P ee057a9504844b4ca8b53f8e4effc594: Log is configured to *not* fsync() on all Append() calls
I20260809 12:22:23.172175 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:22:23.173746 20102 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee057a9504844b4ca8b53f8e4effc594: No bootstrap required, opened a new log
I20260809 12:22:23.183244 20102 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee057a9504844b4ca8b53f8e4effc594 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee057a9504844b4ca8b53f8e4effc594" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36571 } }
I20260809 12:22:23.183928 20102 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee057a9504844b4ca8b53f8e4effc594 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:22:23.184042 20107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:23.176793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39559 (local address 127.19.183.254:36571)
0809 12:22:23.176999 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:23.177006 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:23.177506 (+ 500us) server_negotiation.cc:407] Connection header received
0809 12:22:23.177674 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:23.177680 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:23.177771 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:23.177889 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:23.178411 (+ 522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:23.179609 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:23.181019 (+ 1410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:23.181784 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:23.181920 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:23.183151 (+ 1231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:23.183156 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:23.183160 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:22:23.183180 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:23.183244 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:22:23.183262 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:23.183397 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:23.183609 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 12:22:23.183673 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:22:23.184075 20102 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee057a9504844b4ca8b53f8e4effc594 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee057a9504844b4ca8b53f8e4effc594, State: Initialized, Role: FOLLOWER
I20260809 12:22:23.184892 20102 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee057a9504844b4ca8b53f8e4effc594 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee057a9504844b4ca8b53f8e4effc594" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36571 } }
I20260809 12:22:23.185228 20102 raft_consensus.cc:388] T 00000000000000000000000000000000 P ee057a9504844b4ca8b53f8e4effc594 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:22:23.185392 20102 raft_consensus.cc:482] T 00000000000000000000000000000000 P ee057a9504844b4ca8b53f8e4effc594 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:22:23.185595 20102 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee057a9504844b4ca8b53f8e4effc594 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:22:23.188341 20102 raft_consensus.cc:504] T 00000000000000000000000000000000 P ee057a9504844b4ca8b53f8e4effc594 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee057a9504844b4ca8b53f8e4effc594" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36571 } }
I20260809 12:22:23.188891 20102 leader_election.cc:302] T 00000000000000000000000000000000 P ee057a9504844b4ca8b53f8e4effc594 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ee057a9504844b4ca8b53f8e4effc594; no voters:
I20260809 12:22:23.189580 20102 leader_election.cc:288] T 00000000000000000000000000000000 P ee057a9504844b4ca8b53f8e4effc594 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:22:23.189764 20116 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ee057a9504844b4ca8b53f8e4effc594 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:22:23.190601 20116 raft_consensus.cc:686] T 00000000000000000000000000000000 P ee057a9504844b4ca8b53f8e4effc594 [term 1 LEADER]: Becoming Leader. State: Replica: ee057a9504844b4ca8b53f8e4effc594, State: Running, Role: LEADER
I20260809 12:22:23.191288 20116 consensus_queue.cc:237] T 00000000000000000000000000000000 P ee057a9504844b4ca8b53f8e4effc594 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee057a9504844b4ca8b53f8e4effc594" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36571 } }
I20260809 12:22:23.192024 20102 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee057a9504844b4ca8b53f8e4effc594 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:22:23.195011 20117 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee057a9504844b4ca8b53f8e4effc594 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ee057a9504844b4ca8b53f8e4effc594" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee057a9504844b4ca8b53f8e4effc594" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36571 } } }
I20260809 12:22:23.195135 20118 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee057a9504844b4ca8b53f8e4effc594 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ee057a9504844b4ca8b53f8e4effc594. Latest consensus state: current_term: 1 leader_uuid: "ee057a9504844b4ca8b53f8e4effc594" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ee057a9504844b4ca8b53f8e4effc594" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36571 } } }
I20260809 12:22:23.196069 20117 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee057a9504844b4ca8b53f8e4effc594 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:23.196203 20118 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee057a9504844b4ca8b53f8e4effc594 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:23.199949 20125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:22:23.205577 20125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:22:23.215615 20125 catalog_manager.cc:1347] Generated new cluster ID: b76d7533f69046c088df1e94b6b84faf
I20260809 12:22:23.215761 20125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:22:23.238986 20125 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:22:23.240382 20125 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:22:23.260517 20125 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ee057a9504844b4ca8b53f8e4effc594: Generated new TSK 0
I20260809 12:22:23.261475 20125 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:22:23.276006 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 19990
W20260809 12:22:23.277619 20092 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:57641
W20260809 12:22:23.278474 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 19990: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:22:23.278636 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 19990
W20260809 12:22:23.310830 19969 connection.cc:535] client connection to 127.19.183.254:36571 recv error: Network error: failed to read from TLS socket (remote: 127.19.183.254:36571): Connection reset by peer (error 104)
2026-08-09T12:22:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (2375 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:22:23 dist-test-slave-39l8 krb5kdc[20159](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:22:23 dist-test-slave-39l8 krb5kdc[20159](info): set up 2 sockets
Aug 09 12:22:23 dist-test-slave-39l8 krb5kdc[20159](info): commencing operation
krb5kdc: starting...
W20260809 12:22:24.655618 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.325s user 0.000s sys 0.058s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:22:24 dist-test-slave-39l8 krb5kdc[20159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278144, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:22:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:22:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:22:25.024469 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:36513
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:43021
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:36513
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:22:25.399024 20182 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:22:25.399494 20182 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:22:25.399672 20182 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:22:25.408710 20182 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:22:25.408890 20182 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:22:25.408993 20182 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:22:25.409121 20182 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:22:25.409199 20182 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:22:25.419636 20182 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:43021
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:36513
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:36513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:22:25.421105 20182 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:22:25.422567 20182 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:22:25.437774 20188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:25.437949 20198 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:25.438292 20189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:22:25.438776 20182 server_base.cc:1034] running on GCE node
I20260809 12:22:25.439790 20182 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:22:25.441836 20182 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:22:25.443089 20182 hybrid_clock.cc:648] HybridClock initialized: now 1786278145443027 us; error 50 us; skew 500 ppm
I20260809 12:22:25.443542 20182 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:22:25 dist-test-slave-39l8 krb5kdc[20159](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278145, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:22:25.451534 20182 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:22:25.453266 20182 webserver.cc:460] Webserver started at http://127.19.183.254:36033/ using document root <none> and password file <none>
I20260809 12:22:25.454013 20182 fs_manager.cc:359] Metadata directory not provided
I20260809 12:22:25.454175 20182 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:22:25.454504 20182 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:22:25.457686 20182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "ed7d69d4316f46ab96dfcdefc3e21f2f"
format_stamp: "Formatted at 2026-08-09 12:22:25 on dist-test-slave-39l8"
server_key: "a9536c1cc74d1a708af72763b48f2ba4"
server_key_iv: "857a95478e84740cb4360b47ad7e66f7"
server_key_version: "encryptionkey@0"
I20260809 12:22:25.458503 20182 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "ed7d69d4316f46ab96dfcdefc3e21f2f"
format_stamp: "Formatted at 2026-08-09 12:22:25 on dist-test-slave-39l8"
server_key: "a9536c1cc74d1a708af72763b48f2ba4"
server_key_iv: "857a95478e84740cb4360b47ad7e66f7"
server_key_version: "encryptionkey@0"
I20260809 12:22:25.464319 20182 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 12:22:25.468223 20205 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:22:25.469732 20182 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.003s
I20260809 12:22:25.469933 20182 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "ed7d69d4316f46ab96dfcdefc3e21f2f"
format_stamp: "Formatted at 2026-08-09 12:22:25 on dist-test-slave-39l8"
server_key: "a9536c1cc74d1a708af72763b48f2ba4"
server_key_iv: "857a95478e84740cb4360b47ad7e66f7"
server_key_version: "encryptionkey@0"
I20260809 12:22:25.470168 20182 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:22:25.476379 20182 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:22:25.480929 20182 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:22:25.481267 20182 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:22:25.502002 20182 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:36513
I20260809 12:22:25.502117 20278 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:36513 every 8 connection(s)
I20260809 12:22:25.503573 20182 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:22:25.508797 20280 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:22:25.511565 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 20182
I20260809 12:22:25.511765 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:22:25.512217 20191 external_mini_cluster.cc:1428] Setting key 83794636ed67305aa0dd0d499ea5018e
I20260809 12:22:25.520416 20283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:25.514157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46431 (local address 127.19.183.254:36513)
0809 12:22:25.515025 (+ 868us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:25.515038 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:25.515082 (+ 44us) server_negotiation.cc:407] Connection header received
0809 12:22:25.516804 (+ 1722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:25.516828 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:25.517182 (+ 354us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:25.517606 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:25.518758 (+ 1152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:25.518769 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:25.518787 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:22:25.518814 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:25.518878 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:22:25.518903 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:25.519163 (+ 260us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:25.519299 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 12:22:25.519508 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":662,"thread_start_us":351,"threads_started":1}
I20260809 12:22:25.521083 20280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed7d69d4316f46ab96dfcdefc3e21f2f: Bootstrap starting.
I20260809 12:22:25.524236 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:22:25.524508 20280 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ed7d69d4316f46ab96dfcdefc3e21f2f: Neither blocks nor log segments found. Creating new log.
I20260809 12:22:25.525589 20280 log.cc:824] T 00000000000000000000000000000000 P ed7d69d4316f46ab96dfcdefc3e21f2f: Log is configured to *not* fsync() on all Append() calls
I20260809 12:22:25.528716 20280 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ed7d69d4316f46ab96dfcdefc3e21f2f: No bootstrap required, opened a new log
I20260809 12:22:25.531996 20283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:25.529168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39439 (local address 127.19.183.254:36513)
0809 12:22:25.529365 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:25.529374 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:25.529802 (+ 428us) server_negotiation.cc:407] Connection header received
0809 12:22:25.530016 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:25.530022 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:25.530088 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:25.530204 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:25.531238 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:25.531244 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:25.531249 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:22:25.531271 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:25.531322 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:22:25.531341 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:25.531476 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:25.531613 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 12:22:25.531684 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 12:22:25.537444 20280 raft_consensus.cc:348] T 00000000000000000000000000000000 P ed7d69d4316f46ab96dfcdefc3e21f2f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7d69d4316f46ab96dfcdefc3e21f2f" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36513 } }
I20260809 12:22:25.537900 20280 raft_consensus.cc:374] T 00000000000000000000000000000000 P ed7d69d4316f46ab96dfcdefc3e21f2f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:22:25.538017 20280 raft_consensus.cc:729] T 00000000000000000000000000000000 P ed7d69d4316f46ab96dfcdefc3e21f2f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ed7d69d4316f46ab96dfcdefc3e21f2f, State: Initialized, Role: FOLLOWER
I20260809 12:22:25.538552 20280 consensus_queue.cc:260] T 00000000000000000000000000000000 P ed7d69d4316f46ab96dfcdefc3e21f2f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7d69d4316f46ab96dfcdefc3e21f2f" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36513 } }
I20260809 12:22:25.538844 20280 raft_consensus.cc:388] T 00000000000000000000000000000000 P ed7d69d4316f46ab96dfcdefc3e21f2f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:22:25.538955 20280 raft_consensus.cc:482] T 00000000000000000000000000000000 P ed7d69d4316f46ab96dfcdefc3e21f2f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:22:25.539083 20280 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ed7d69d4316f46ab96dfcdefc3e21f2f [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:22:25.541265 20280 raft_consensus.cc:504] T 00000000000000000000000000000000 P ed7d69d4316f46ab96dfcdefc3e21f2f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7d69d4316f46ab96dfcdefc3e21f2f" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36513 } }
I20260809 12:22:25.541733 20280 leader_election.cc:302] T 00000000000000000000000000000000 P ed7d69d4316f46ab96dfcdefc3e21f2f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ed7d69d4316f46ab96dfcdefc3e21f2f; no voters:
I20260809 12:22:25.542356 20280 leader_election.cc:288] T 00000000000000000000000000000000 P ed7d69d4316f46ab96dfcdefc3e21f2f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:22:25.542513 20294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ed7d69d4316f46ab96dfcdefc3e21f2f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:22:25.543293 20294 raft_consensus.cc:686] T 00000000000000000000000000000000 P ed7d69d4316f46ab96dfcdefc3e21f2f [term 1 LEADER]: Becoming Leader. State: Replica: ed7d69d4316f46ab96dfcdefc3e21f2f, State: Running, Role: LEADER
I20260809 12:22:25.543874 20294 consensus_queue.cc:237] T 00000000000000000000000000000000 P ed7d69d4316f46ab96dfcdefc3e21f2f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7d69d4316f46ab96dfcdefc3e21f2f" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36513 } }
I20260809 12:22:25.544696 20280 sys_catalog.cc:564] T 00000000000000000000000000000000 P ed7d69d4316f46ab96dfcdefc3e21f2f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:22:25.547152 20295 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed7d69d4316f46ab96dfcdefc3e21f2f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ed7d69d4316f46ab96dfcdefc3e21f2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7d69d4316f46ab96dfcdefc3e21f2f" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36513 } } }
I20260809 12:22:25.547617 20295 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed7d69d4316f46ab96dfcdefc3e21f2f [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:25.547438 20296 sys_catalog.cc:455] T 00000000000000000000000000000000 P ed7d69d4316f46ab96dfcdefc3e21f2f [sys.catalog]: SysCatalogTable state changed. Reason: New leader ed7d69d4316f46ab96dfcdefc3e21f2f. Latest consensus state: current_term: 1 leader_uuid: "ed7d69d4316f46ab96dfcdefc3e21f2f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ed7d69d4316f46ab96dfcdefc3e21f2f" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 36513 } } }
I20260809 12:22:25.548125 20296 sys_catalog.cc:458] T 00000000000000000000000000000000 P ed7d69d4316f46ab96dfcdefc3e21f2f [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:25.549356 20304 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:22:25.554979 20304 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:22:25.564109 20304 catalog_manager.cc:1347] Generated new cluster ID: ce2e51f8619647c0b75d62aa33c6c2c2
I20260809 12:22:25.564248 20304 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:22:25.584043 20304 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:22:25.584575 20304 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:22:25.592252 20304 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ed7d69d4316f46ab96dfcdefc3e21f2f: Generated new TSK 0
I20260809 12:22:25.593312 20304 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:22:25.611161 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 20182
W20260809 12:22:25.612461 20273 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:46431
W20260809 12:22:25.613297 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 20182: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:22:25.613447 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 20182
2026-08-09T12:22:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (2331 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:22:25 dist-test-slave-39l8 krb5kdc[20323](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:22:25 dist-test-slave-39l8 krb5kdc[20323](info): set up 2 sockets
Aug 09 12:22:25 dist-test-slave-39l8 krb5kdc[20323](info): commencing operation
krb5kdc: starting...
W20260809 12:22:26.972070 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.311s user 0.004s sys 0.059s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:22:27 dist-test-slave-39l8 krb5kdc[20323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:22:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:22:27Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:22:27.324098 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:38663
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:41481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:38663
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:22:27.674095 20343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:22:27.674443 20343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:22:27.674636 20343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:22:27.681746 20343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:22:27.681857 20343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:22:27.681905 20343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:22:27.681958 20343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:22:27.682056 20343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:22:27.691632 20343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:41481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:38663
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:38663
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:22:27.692853 20343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:22:27.694216 20343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:22:27.706102 20351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:27.706137 20352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:27.706163 20354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:22:27.707269 20343 server_base.cc:1034] running on GCE node
I20260809 12:22:27.708002 20343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:22:27.709304 20343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:22:27.710510 20343 hybrid_clock.cc:648] HybridClock initialized: now 1786278147710444 us; error 49 us; skew 500 ppm
I20260809 12:22:27.710906 20343 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:22:27 dist-test-slave-39l8 krb5kdc[20323](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278147, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:22:27.713976 20343 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:22:27.715431 20343 webserver.cc:460] Webserver started at http://127.19.183.254:42843/ using document root <none> and password file <none>
I20260809 12:22:27.716055 20343 fs_manager.cc:359] Metadata directory not provided
I20260809 12:22:27.716173 20343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:22:27.716418 20343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:22:27.719308 20343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "1daf0901895c417598c8d9b7d9fe6799"
format_stamp: "Formatted at 2026-08-09 12:22:27 on dist-test-slave-39l8"
server_key: "240a38c4ed89e13d3fcc1ca63a58d5d5"
server_key_iv: "9805fabcd9c46818eb7b9e51d966ee99"
server_key_version: "encryptionkey@0"
I20260809 12:22:27.720039 20343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "1daf0901895c417598c8d9b7d9fe6799"
format_stamp: "Formatted at 2026-08-09 12:22:27 on dist-test-slave-39l8"
server_key: "240a38c4ed89e13d3fcc1ca63a58d5d5"
server_key_iv: "9805fabcd9c46818eb7b9e51d966ee99"
server_key_version: "encryptionkey@0"
I20260809 12:22:27.724900 20343 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.004s
I20260809 12:22:27.728248 20366 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:22:27.729519 20343 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260809 12:22:27.729732 20343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "1daf0901895c417598c8d9b7d9fe6799"
format_stamp: "Formatted at 2026-08-09 12:22:27 on dist-test-slave-39l8"
server_key: "240a38c4ed89e13d3fcc1ca63a58d5d5"
server_key_iv: "9805fabcd9c46818eb7b9e51d966ee99"
server_key_version: "encryptionkey@0"
I20260809 12:22:27.729933 20343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:22:27.747839 20343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:22:27.751698 20343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:22:27.751974 20343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:22:27.770098 20343 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:38663
I20260809 12:22:27.770138 20439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:38663 every 8 connection(s)
I20260809 12:22:27.771418 20343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:22:27.775005 20440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:22:27.777981 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 20343
I20260809 12:22:27.778182 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:22:27.778734 20191 external_mini_cluster.cc:1428] Setting key 0e2012eec7a3cb1715e6368c1072ffff
I20260809 12:22:27.787118 20440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1daf0901895c417598c8d9b7d9fe6799: Bootstrap starting.
I20260809 12:22:27.790925 20440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1daf0901895c417598c8d9b7d9fe6799: Neither blocks nor log segments found. Creating new log.
I20260809 12:22:27.792277 20440 log.cc:824] T 00000000000000000000000000000000 P 1daf0901895c417598c8d9b7d9fe6799: Log is configured to *not* fsync() on all Append() calls
Aug 09 12:22:27 dist-test-slave-39l8 krb5kdc[20323](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278147, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:22:27.795226 20440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1daf0901895c417598c8d9b7d9fe6799: No bootstrap required, opened a new log
I20260809 12:22:27.802066 20446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:27.780553 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41835 (local address 127.19.183.254:38663)
0809 12:22:27.781330 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:27.781341 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:27.781394 (+ 53us) server_negotiation.cc:407] Connection header received
0809 12:22:27.783083 (+ 1689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:27.783104 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:27.783491 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:27.783804 (+ 313us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:27.785589 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:27.786976 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:27.788904 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:27.789578 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:27.789699 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:27.794856 (+ 5157us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:27.794864 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:27.794877 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:27.794914 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:27.798103 (+ 3189us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:27.798794 (+ 691us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:27.798802 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:27.798807 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:27.798998 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:27.799403 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:27.799410 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:27.799418 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:27.799910 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:27.800278 (+ 368us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:27.800719 (+ 441us) server_negotiation.cc:299] Negotiation successful
0809 12:22:27.801122 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":588,"thread_start_us":378,"threads_started":1}
I20260809 12:22:27.803169 20440 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1daf0901895c417598c8d9b7d9fe6799 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1daf0901895c417598c8d9b7d9fe6799" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38663 } }
I20260809 12:22:27.803638 20440 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1daf0901895c417598c8d9b7d9fe6799 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:22:27.803756 20440 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1daf0901895c417598c8d9b7d9fe6799 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1daf0901895c417598c8d9b7d9fe6799, State: Initialized, Role: FOLLOWER
I20260809 12:22:27.804382 20440 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1daf0901895c417598c8d9b7d9fe6799 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1daf0901895c417598c8d9b7d9fe6799" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38663 } }
I20260809 12:22:27.804733 20440 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1daf0901895c417598c8d9b7d9fe6799 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:22:27.804880 20440 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1daf0901895c417598c8d9b7d9fe6799 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:22:27.805068 20440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1daf0901895c417598c8d9b7d9fe6799 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:22:27.806068 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:22:27.807657 20440 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1daf0901895c417598c8d9b7d9fe6799 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1daf0901895c417598c8d9b7d9fe6799" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38663 } }
I20260809 12:22:27.808135 20440 leader_election.cc:302] T 00000000000000000000000000000000 P 1daf0901895c417598c8d9b7d9fe6799 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1daf0901895c417598c8d9b7d9fe6799; no voters:
I20260809 12:22:27.808815 20440 leader_election.cc:288] T 00000000000000000000000000000000 P 1daf0901895c417598c8d9b7d9fe6799 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:22:27.808910 20450 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1daf0901895c417598c8d9b7d9fe6799 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:22:27.810228 20450 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1daf0901895c417598c8d9b7d9fe6799 [term 1 LEADER]: Becoming Leader. State: Replica: 1daf0901895c417598c8d9b7d9fe6799, State: Running, Role: LEADER
I20260809 12:22:27.810885 20450 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1daf0901895c417598c8d9b7d9fe6799 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1daf0901895c417598c8d9b7d9fe6799" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38663 } }
I20260809 12:22:27.811187 20440 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1daf0901895c417598c8d9b7d9fe6799 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:22:27.813839 20456 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1daf0901895c417598c8d9b7d9fe6799 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1daf0901895c417598c8d9b7d9fe6799. Latest consensus state: current_term: 1 leader_uuid: "1daf0901895c417598c8d9b7d9fe6799" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1daf0901895c417598c8d9b7d9fe6799" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38663 } } }
I20260809 12:22:27.814285 20456 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1daf0901895c417598c8d9b7d9fe6799 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:27.814971 20452 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1daf0901895c417598c8d9b7d9fe6799 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1daf0901895c417598c8d9b7d9fe6799" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1daf0901895c417598c8d9b7d9fe6799" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 38663 } } }
I20260809 12:22:27.821005 20452 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1daf0901895c417598c8d9b7d9fe6799 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:27.824124 20464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:22:27.830843 20464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:22:27.834825 20446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:27.814739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60721 (local address 127.19.183.254:38663)
0809 12:22:27.815417 (+ 678us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:27.815425 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:27.817421 (+ 1996us) server_negotiation.cc:407] Connection header received
0809 12:22:27.817919 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:27.817928 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:27.818026 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:27.818125 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:27.821417 (+ 3292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:27.822896 (+ 1479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:27.824594 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:27.825060 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:27.825192 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:27.828082 (+ 2890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:27.828091 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:27.828096 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:27.828133 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:27.831668 (+ 3535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:27.832336 (+ 668us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:27.832342 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:27.832345 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:27.832501 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:27.832916 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:27.832920 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:27.832922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:27.833402 (+ 480us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:27.833671 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:27.834049 (+ 378us) server_negotiation.cc:299] Negotiation successful
0809 12:22:27.834269 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":513}
I20260809 12:22:27.845379 20464 catalog_manager.cc:1347] Generated new cluster ID: 9ddf27bd223f4a7aa8a356693770b8b6
I20260809 12:22:27.845490 20464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:22:27.873523 20464 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:22:27.875531 20464 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:22:27.884673 20464 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1daf0901895c417598c8d9b7d9fe6799: Generated new TSK 0
I20260809 12:22:27.885577 20464 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:22:27.892894 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 20343
I20260809 12:22:27.893965 20435 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:22:27.986991 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 20343
2026-08-09T12:22:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (2367 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:22:28 dist-test-slave-39l8 krb5kdc[20487](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:22:28 dist-test-slave-39l8 krb5kdc[20487](info): set up 2 sockets
Aug 09 12:22:28 dist-test-slave-39l8 krb5kdc[20487](info): commencing operation
krb5kdc: starting...
W20260809 12:22:29.303683 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.275s user 0.000s sys 0.052s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:22:29 dist-test-slave-39l8 krb5kdc[20487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278149, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:22:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:22:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:22:29.624081 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:33999
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:41473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:33999
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:22:29.958593 20507 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:22:29.958886 20507 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:22:29.959084 20507 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:22:29.965945 20507 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:22:29.966056 20507 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:22:29.966176 20507 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:22:29.966275 20507 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:22:29.966356 20507 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:22:29.977103 20507 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:41473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:33999
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:33999
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:22:29.978395 20507 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:22:29.979718 20507 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:22:29.991638 20513 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:29.991652 20514 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:29.991676 20516 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:22:29.992198 20507 server_base.cc:1034] running on GCE node
I20260809 12:22:29.993052 20507 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:22:29.994408 20507 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:22:29.995641 20507 hybrid_clock.cc:648] HybridClock initialized: now 1786278149995590 us; error 59 us; skew 500 ppm
I20260809 12:22:29.995988 20507 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:22:29 dist-test-slave-39l8 krb5kdc[20487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278149, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:22:29.999156 20507 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:22:30.000684 20507 webserver.cc:460] Webserver started at http://127.19.183.254:46261/ using document root <none> and password file <none>
I20260809 12:22:30.001425 20507 fs_manager.cc:359] Metadata directory not provided
I20260809 12:22:30.001576 20507 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:22:30.001899 20507 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:22:30.004974 20507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "b1c0a6e9534543ca91fcdf769bd266d6"
format_stamp: "Formatted at 2026-08-09 12:22:29 on dist-test-slave-39l8"
server_key: "54a753febb4fc1df2956f2fc03553ab9"
server_key_iv: "9146d0e088131f84652917263de51ea2"
server_key_version: "encryptionkey@0"
I20260809 12:22:30.005786 20507 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "b1c0a6e9534543ca91fcdf769bd266d6"
format_stamp: "Formatted at 2026-08-09 12:22:29 on dist-test-slave-39l8"
server_key: "54a753febb4fc1df2956f2fc03553ab9"
server_key_iv: "9146d0e088131f84652917263de51ea2"
server_key_version: "encryptionkey@0"
I20260809 12:22:30.011114 20507 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 12:22:30.016477 20526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:22:30.018283 20507 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260809 12:22:30.018509 20507 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "b1c0a6e9534543ca91fcdf769bd266d6"
format_stamp: "Formatted at 2026-08-09 12:22:29 on dist-test-slave-39l8"
server_key: "54a753febb4fc1df2956f2fc03553ab9"
server_key_iv: "9146d0e088131f84652917263de51ea2"
server_key_version: "encryptionkey@0"
I20260809 12:22:30.018795 20507 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:22:30.040330 20507 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:22:30.044283 20507 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:22:30.044579 20507 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:22:30.063670 20507 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:33999
I20260809 12:22:30.063704 20601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:33999 every 8 connection(s)
I20260809 12:22:30.065109 20507 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:22:30.068929 20603 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:22:30.072050 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 20507
I20260809 12:22:30.072243 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:22:30.072686 20191 external_mini_cluster.cc:1428] Setting key 7e8d79d49165ebf5037cd8d6297f1093
I20260809 12:22:30.082448 20603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1c0a6e9534543ca91fcdf769bd266d6: Bootstrap starting.
I20260809 12:22:30.086526 20603 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b1c0a6e9534543ca91fcdf769bd266d6: Neither blocks nor log segments found. Creating new log.
I20260809 12:22:30.087915 20603 log.cc:824] T 00000000000000000000000000000000 P b1c0a6e9534543ca91fcdf769bd266d6: Log is configured to *not* fsync() on all Append() calls
I20260809 12:22:30.088080 20608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:30.074649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36303 (local address 127.19.183.254:33999)
0809 12:22:30.075483 (+ 834us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:30.075498 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:30.075548 (+ 50us) server_negotiation.cc:407] Connection header received
0809 12:22:30.077887 (+ 2339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:30.077917 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:30.078450 (+ 533us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:30.078916 (+ 466us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:30.079433 (+ 517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:30.081475 (+ 2042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:30.082912 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:30.083904 (+ 992us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:30.084131 (+ 227us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:30.085229 (+ 1098us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:30.085241 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:30.085259 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:22:30.085981 (+ 722us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:30.086072 (+ 91us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:22:30.086109 (+ 37us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:30.086236 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:30.086389 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 12:22:30.086683 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":597,"thread_start_us":327,"threads_started":1}
I20260809 12:22:30.091137 20603 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b1c0a6e9534543ca91fcdf769bd266d6: No bootstrap required, opened a new log
I20260809 12:22:30.092043 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:22:30.099931 20603 raft_consensus.cc:348] T 00000000000000000000000000000000 P b1c0a6e9534543ca91fcdf769bd266d6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1c0a6e9534543ca91fcdf769bd266d6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33999 } }
I20260809 12:22:30.100430 20603 raft_consensus.cc:374] T 00000000000000000000000000000000 P b1c0a6e9534543ca91fcdf769bd266d6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:22:30.100543 20603 raft_consensus.cc:729] T 00000000000000000000000000000000 P b1c0a6e9534543ca91fcdf769bd266d6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b1c0a6e9534543ca91fcdf769bd266d6, State: Initialized, Role: FOLLOWER
I20260809 12:22:30.101186 20603 consensus_queue.cc:260] T 00000000000000000000000000000000 P b1c0a6e9534543ca91fcdf769bd266d6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1c0a6e9534543ca91fcdf769bd266d6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33999 } }
I20260809 12:22:30.101512 20603 raft_consensus.cc:388] T 00000000000000000000000000000000 P b1c0a6e9534543ca91fcdf769bd266d6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:22:30.101666 20603 raft_consensus.cc:482] T 00000000000000000000000000000000 P b1c0a6e9534543ca91fcdf769bd266d6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:22:30.101835 20603 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b1c0a6e9534543ca91fcdf769bd266d6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:22:30.103258 20608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:30.096238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36683 (local address 127.19.183.254:33999)
0809 12:22:30.096469 (+ 231us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:30.096478 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:30.096982 (+ 504us) server_negotiation.cc:407] Connection header received
0809 12:22:30.097151 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:30.097155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:30.097212 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:30.097290 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:30.097888 (+ 598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:30.098591 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:30.100470 (+ 1879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:30.101162 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:30.101263 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:30.102474 (+ 1211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:30.102479 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:30.102485 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:22:30.102502 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:30.102542 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:22:30.102560 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:30.102639 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:30.102829 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 12:22:30.102902 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260809 12:22:30.104827 20603 raft_consensus.cc:504] T 00000000000000000000000000000000 P b1c0a6e9534543ca91fcdf769bd266d6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1c0a6e9534543ca91fcdf769bd266d6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33999 } }
I20260809 12:22:30.105329 20603 leader_election.cc:302] T 00000000000000000000000000000000 P b1c0a6e9534543ca91fcdf769bd266d6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b1c0a6e9534543ca91fcdf769bd266d6; no voters:
I20260809 12:22:30.105877 20603 leader_election.cc:288] T 00000000000000000000000000000000 P b1c0a6e9534543ca91fcdf769bd266d6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:22:30.105997 20620 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b1c0a6e9534543ca91fcdf769bd266d6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:22:30.106736 20620 raft_consensus.cc:686] T 00000000000000000000000000000000 P b1c0a6e9534543ca91fcdf769bd266d6 [term 1 LEADER]: Becoming Leader. State: Replica: b1c0a6e9534543ca91fcdf769bd266d6, State: Running, Role: LEADER
I20260809 12:22:30.107280 20620 consensus_queue.cc:237] T 00000000000000000000000000000000 P b1c0a6e9534543ca91fcdf769bd266d6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1c0a6e9534543ca91fcdf769bd266d6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33999 } }
I20260809 12:22:30.108074 20603 sys_catalog.cc:564] T 00000000000000000000000000000000 P b1c0a6e9534543ca91fcdf769bd266d6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:22:30.110383 20622 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1c0a6e9534543ca91fcdf769bd266d6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b1c0a6e9534543ca91fcdf769bd266d6. Latest consensus state: current_term: 1 leader_uuid: "b1c0a6e9534543ca91fcdf769bd266d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1c0a6e9534543ca91fcdf769bd266d6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33999 } } }
I20260809 12:22:30.110862 20622 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1c0a6e9534543ca91fcdf769bd266d6 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:30.110924 20621 sys_catalog.cc:455] T 00000000000000000000000000000000 P b1c0a6e9534543ca91fcdf769bd266d6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b1c0a6e9534543ca91fcdf769bd266d6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b1c0a6e9534543ca91fcdf769bd266d6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33999 } } }
I20260809 12:22:30.111330 20621 sys_catalog.cc:458] T 00000000000000000000000000000000 P b1c0a6e9534543ca91fcdf769bd266d6 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:30.113075 20629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:22:30.118371 20629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:22:30.126912 20629 catalog_manager.cc:1347] Generated new cluster ID: 6ed80c395b1b4c7490703dec644b30bb
I20260809 12:22:30.127040 20629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:22:30.138566 20629 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:22:30.139851 20629 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:22:30.145157 20629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b1c0a6e9534543ca91fcdf769bd266d6: Generated new TSK 0
I20260809 12:22:30.145911 20629 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:22:30.156211 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 20507
W20260809 12:22:30.157572 20597 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:36303
W20260809 12:22:30.158275 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 20507: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:22:30.158427 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 20507
2026-08-09T12:22:30Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (2173 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:22:30 dist-test-slave-39l8 krb5kdc[20658](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:22:30 dist-test-slave-39l8 krb5kdc[20658](info): set up 2 sockets
Aug 09 12:22:30 dist-test-slave-39l8 krb5kdc[20658](info): commencing operation
krb5kdc: starting...
W20260809 12:22:31.474294 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.271s user 0.002s sys 0.048s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:22:31 dist-test-slave-39l8 krb5kdc[20658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:22:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:22:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:22:31.791150 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:33969
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:42525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:33969
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:22:32.139266 20678 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:22:32.139551 20678 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:22:32.139642 20678 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:22:32.146514 20678 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:22:32.146639 20678 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:22:32.146685 20678 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:22:32.146755 20678 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:22:32.146792 20678 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:22:32.156293 20678 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:42525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:33969
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:33969
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:22:32.157506 20678 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:22:32.158767 20678 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:22:32.170727 20684 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:32.170670 20685 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:32.170794 20691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:22:32.171491 20678 server_base.cc:1034] running on GCE node
I20260809 12:22:32.172322 20678 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:22:32.173683 20678 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:22:32.174916 20678 hybrid_clock.cc:648] HybridClock initialized: now 1786278152174873 us; error 55 us; skew 500 ppm
I20260809 12:22:32.175263 20678 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:22:32 dist-test-slave-39l8 krb5kdc[20658](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278152, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:22:32.181432 20678 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:22:32.183122 20678 webserver.cc:460] Webserver started at http://127.19.183.254:34353/ using document root <none> and password file <none>
I20260809 12:22:32.183820 20678 fs_manager.cc:359] Metadata directory not provided
I20260809 12:22:32.183969 20678 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:22:32.184288 20678 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:22:32.187348 20678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "61bbd808d9bb432c804a6d6165eb739b"
format_stamp: "Formatted at 2026-08-09 12:22:32 on dist-test-slave-39l8"
server_key: "432d746ebcbb7194252f0f3f13c4735f"
server_key_iv: "bcf45fe55f49c510bddc9afbbee1ecfd"
server_key_version: "encryptionkey@0"
I20260809 12:22:32.188158 20678 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "61bbd808d9bb432c804a6d6165eb739b"
format_stamp: "Formatted at 2026-08-09 12:22:32 on dist-test-slave-39l8"
server_key: "432d746ebcbb7194252f0f3f13c4735f"
server_key_iv: "bcf45fe55f49c510bddc9afbbee1ecfd"
server_key_version: "encryptionkey@0"
I20260809 12:22:32.193295 20678 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260809 12:22:32.196694 20702 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:22:32.197932 20678 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260809 12:22:32.198132 20678 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "61bbd808d9bb432c804a6d6165eb739b"
format_stamp: "Formatted at 2026-08-09 12:22:32 on dist-test-slave-39l8"
server_key: "432d746ebcbb7194252f0f3f13c4735f"
server_key_iv: "bcf45fe55f49c510bddc9afbbee1ecfd"
server_key_version: "encryptionkey@0"
I20260809 12:22:32.198335 20678 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:22:32.203917 20678 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:22:32.207772 20678 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:22:32.208062 20678 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:22:32.226884 20678 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:33969
I20260809 12:22:32.226902 20775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:33969 every 8 connection(s)
I20260809 12:22:32.228278 20678 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:22:32.232013 20776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:22:32.233777 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 20678
I20260809 12:22:32.233971 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:22:32.234496 20191 external_mini_cluster.cc:1428] Setting key 69075e4496915bbe0f05251539ee5975
I20260809 12:22:32.242107 20779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:32.236322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39123 (local address 127.19.183.254:33969)
0809 12:22:32.237124 (+ 802us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:32.237138 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:32.237180 (+ 42us) server_negotiation.cc:407] Connection header received
0809 12:22:32.238800 (+ 1620us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:32.238823 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:32.239160 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:32.239470 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:32.240555 (+ 1085us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:32.240561 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:32.240574 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:22:32.240588 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:32.240657 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:22:32.240676 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:32.240840 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:32.241008 (+ 168us) server_negotiation.cc:299] Negotiation successful
0809 12:22:32.241208 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":585,"thread_start_us":363,"threads_started":1}
I20260809 12:22:32.245358 20776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61bbd808d9bb432c804a6d6165eb739b: Bootstrap starting.
I20260809 12:22:32.245781 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:22:32.249194 20776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61bbd808d9bb432c804a6d6165eb739b: Neither blocks nor log segments found. Creating new log.
I20260809 12:22:32.250600 20776 log.cc:824] T 00000000000000000000000000000000 P 61bbd808d9bb432c804a6d6165eb739b: Log is configured to *not* fsync() on all Append() calls
I20260809 12:22:32.253830 20776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61bbd808d9bb432c804a6d6165eb739b: No bootstrap required, opened a new log
I20260809 12:22:32.253998 20779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:32.251106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44187 (local address 127.19.183.254:33969)
0809 12:22:32.251338 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:32.251351 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:32.251854 (+ 503us) server_negotiation.cc:407] Connection header received
0809 12:22:32.252056 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:32.252061 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:32.252112 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:32.252225 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:32.253274 (+ 1049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:32.253279 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:32.253283 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 12:22:32.253301 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:32.253349 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 12:22:32.253371 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:32.253490 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:32.253616 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 12:22:32.253673 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 12:22:32.261327 20776 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61bbd808d9bb432c804a6d6165eb739b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61bbd808d9bb432c804a6d6165eb739b" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33969 } }
I20260809 12:22:32.261686 20776 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61bbd808d9bb432c804a6d6165eb739b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 12:22:32.261817 20776 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61bbd808d9bb432c804a6d6165eb739b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61bbd808d9bb432c804a6d6165eb739b, State: Initialized, Role: FOLLOWER
I20260809 12:22:32.262380 20776 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61bbd808d9bb432c804a6d6165eb739b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61bbd808d9bb432c804a6d6165eb739b" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33969 } }
I20260809 12:22:32.262614 20776 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61bbd808d9bb432c804a6d6165eb739b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 12:22:32.262810 20776 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61bbd808d9bb432c804a6d6165eb739b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 12:22:32.262995 20776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61bbd808d9bb432c804a6d6165eb739b [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:22:32.265129 20776 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61bbd808d9bb432c804a6d6165eb739b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61bbd808d9bb432c804a6d6165eb739b" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33969 } }
I20260809 12:22:32.265712 20776 leader_election.cc:302] T 00000000000000000000000000000000 P 61bbd808d9bb432c804a6d6165eb739b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61bbd808d9bb432c804a6d6165eb739b; no voters:
I20260809 12:22:32.266423 20776 leader_election.cc:288] T 00000000000000000000000000000000 P 61bbd808d9bb432c804a6d6165eb739b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 12:22:32.266583 20790 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61bbd808d9bb432c804a6d6165eb739b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:22:32.267302 20790 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61bbd808d9bb432c804a6d6165eb739b [term 1 LEADER]: Becoming Leader. State: Replica: 61bbd808d9bb432c804a6d6165eb739b, State: Running, Role: LEADER
I20260809 12:22:32.267838 20790 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61bbd808d9bb432c804a6d6165eb739b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61bbd808d9bb432c804a6d6165eb739b" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33969 } }
I20260809 12:22:32.268513 20776 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61bbd808d9bb432c804a6d6165eb739b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:22:32.270800 20791 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61bbd808d9bb432c804a6d6165eb739b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61bbd808d9bb432c804a6d6165eb739b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61bbd808d9bb432c804a6d6165eb739b" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33969 } } }
I20260809 12:22:32.271040 20792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61bbd808d9bb432c804a6d6165eb739b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61bbd808d9bb432c804a6d6165eb739b. Latest consensus state: current_term: 1 leader_uuid: "61bbd808d9bb432c804a6d6165eb739b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61bbd808d9bb432c804a6d6165eb739b" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33969 } } }
I20260809 12:22:32.271301 20791 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61bbd808d9bb432c804a6d6165eb739b [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:32.271351 20792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61bbd808d9bb432c804a6d6165eb739b [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:32.273155 20801 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:22:32.278730 20801 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:22:32.287129 20801 catalog_manager.cc:1347] Generated new cluster ID: b3e349587fd04c998577f84c39b5358e
I20260809 12:22:32.287235 20801 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:22:32.297020 20801 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:22:32.297487 20801 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:22:32.311201 20801 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61bbd808d9bb432c804a6d6165eb739b: Generated new TSK 0
I20260809 12:22:32.312167 20801 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:22:32.322726 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 20678
W20260809 12:22:32.323807 20770 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.CheckLeaks from {username='slave'} at 127.0.0.1:39123
W20260809 12:22:32.324498 20191 external_mini_cluster.cc:1652] Unable to check leaks on /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 20678: Remote error: Not authorized: unauthorized access to method: CheckLeaks
I20260809 12:22:32.324658 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 20678
2026-08-09T12:22:32Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (2169 ms)
[----------] 18 tests from AuthTokenIssuingTest (40185 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:22:32 dist-test-slave-39l8 krb5kdc[20819](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:22:32 dist-test-slave-39l8 krb5kdc[20819](info): set up 2 sockets
Aug 09 12:22:32 dist-test-slave-39l8 krb5kdc[20819](info): commencing operation
krb5kdc: starting...
W20260809 12:22:33.620810 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.247s user 0.000s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:22:33 dist-test-slave-39l8 krb5kdc[20819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:22:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:22:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:22:33.965607 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:33983
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:35929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:33983,127.19.183.253:43517,127.19.183.252:37575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:22:34.318050 20834 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 12:22:34.318361 20834 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:22:34.318435 20834 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:22:34.318517 20834 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:22:34.325788 20834 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:22:34.325883 20834 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:22:34.325942 20834 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:22:34.325994 20834 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:22:34.326056 20834 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:22:34.335817 20834 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:35929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:33983,127.19.183.253:43517,127.19.183.252:37575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:33983
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:22:34.337086 20834 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:22:34.338359 20834 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:22:34.350175 20843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:34.350385 20844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:34.355020 20846 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:22:34.356071 20834 server_base.cc:1034] running on GCE node
I20260809 12:22:34.356880 20834 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:22:34.358335 20834 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:22:34.359596 20834 hybrid_clock.cc:648] HybridClock initialized: now 1786278154359522 us; error 49 us; skew 500 ppm
I20260809 12:22:34.359933 20834 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:22:34 dist-test-slave-39l8 krb5kdc[20819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278154, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:22:34.363359 20834 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:22:34.364797 20834 webserver.cc:460] Webserver started at http://127.19.183.254:44659/ using document root <none> and password file <none>
I20260809 12:22:34.365520 20834 fs_manager.cc:359] Metadata directory not provided
I20260809 12:22:34.365659 20834 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:22:34.366006 20834 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:22:34.369113 20834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "f08f78843868434f9b019cf28c5039f6"
format_stamp: "Formatted at 2026-08-09 12:22:34 on dist-test-slave-39l8"
server_key: "74a1145d378ae0c593c93e26a8a82400"
server_key_iv: "cf41a100fed48a876ddf9e8b1c6b6cde"
server_key_version: "encryptionkey@0"
I20260809 12:22:34.369946 20834 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "f08f78843868434f9b019cf28c5039f6"
format_stamp: "Formatted at 2026-08-09 12:22:34 on dist-test-slave-39l8"
server_key: "74a1145d378ae0c593c93e26a8a82400"
server_key_iv: "cf41a100fed48a876ddf9e8b1c6b6cde"
server_key_version: "encryptionkey@0"
I20260809 12:22:34.375200 20834 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.001s
I20260809 12:22:34.378803 20854 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:22:34.380067 20834 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260809 12:22:34.380249 20834 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "f08f78843868434f9b019cf28c5039f6"
format_stamp: "Formatted at 2026-08-09 12:22:34 on dist-test-slave-39l8"
server_key: "74a1145d378ae0c593c93e26a8a82400"
server_key_iv: "cf41a100fed48a876ddf9e8b1c6b6cde"
server_key_version: "encryptionkey@0"
I20260809 12:22:34.380467 20834 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:22:34.398298 20834 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:22:34.402791 20834 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:22:34.403080 20834 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:22:34.422408 20834 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:33983
I20260809 12:22:34.422442 20928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:33983 every 8 connection(s)
I20260809 12:22:34.423785 20834 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:22:34.427462 20930 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:22:34.432035 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 20834
I20260809 12:22:34.432273 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:22:34.432911 20191 external_mini_cluster.cc:1428] Setting key 5e8b3e771da0caefb9e3140c82820e2a
I20260809 12:22:34.434113 20930 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33983 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:22:34 dist-test-slave-39l8 krb5kdc[20819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278154, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:22:34.458822 20933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:34.437221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38590 (local address 127.19.183.254:33983)
0809 12:22:34.442810 (+ 5589us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:34.442816 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:34.442847 (+ 31us) server_negotiation.cc:407] Connection header received
0809 12:22:34.442977 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:34.442982 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:34.443205 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:34.443420 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:34.444739 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:34.445926 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:34.447365 (+ 1439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:34.447898 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:34.448116 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:34.451588 (+ 3472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:34.451598 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:34.451609 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:34.451644 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:34.454370 (+ 2726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:34.455140 (+ 770us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:34.455146 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:34.455148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:34.455292 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:34.455696 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:34.455705 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:34.455707 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:34.456166 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:34.456374 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:34.456916 (+ 542us) server_negotiation.cc:299] Negotiation successful
0809 12:22:34.457324 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":38,"server-negotiator.queue_time_us":5349,"spinlock_wait_cycles":18304,"thread_start_us":5148,"threads_started":1}
I20260809 12:22:34.458817 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:34.437222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:33983 (local address 127.0.0.1:38590)
0809 12:22:34.438224 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:34.438286 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 12:22:34.441848 (+ 3562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:22:34.443434 (+ 1586us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:22:34.443441 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:22:34.444155 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:22:34.444493 (+ 338us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:34.444510 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:34.446084 (+ 1574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:34.446094 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:34.447213 (+ 1119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:34.447226 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:34.448136 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:34.448143 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:34.448316 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:34.448988 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:22:34.449012 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:22:34.451364 (+ 2352us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:22:34.454509 (+ 3145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:34.454520 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:34.454531 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:34.454991 (+ 460us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:34.455407 (+ 416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:34.455413 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:34.455416 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:34.455569 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:34.456311 (+ 742us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:22:34.456319 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:22:34.456534 (+ 215us) client_negotiation.cc:769] Sending connection context
0809 12:22:34.456775 (+ 241us) client_negotiation.cc:240] Negotiation successful
0809 12:22:34.457439 (+ 664us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":641,"mutex_wait_us":202,"thread_start_us":393,"threads_started":1}
I20260809 12:22:34.462215 20930 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:22:34.463543 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:34.463020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:43517 (local address 127.0.0.1:37386)
0809 12:22:34.463250 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:34.463383 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260809 12:22:34.463879 20858 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.183.253:43517: connect: Connection refused (error 111)
I20260809 12:22:34.465574 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:34.465116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:43517 (local address 127.0.0.1:37394)
0809 12:22:34.465292 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:34.465387 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260809 12:22:34.465797 20930 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.253:43517: Network error: Client connection negotiation failed: client connection to 127.19.183.253:43517: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.19.183.253@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.253.keytab.
Entry for principal kudu/127.19.183.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.253.keytab.
I20260809 12:22:34.507092 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.253:43517
--webserver_interface=127.19.183.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.253.keytab
--principal=kudu/127.19.183.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:35929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:33983,127.19.183.253:43517,127.19.183.252:37575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:22:34.515158 20930 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } attempt: 1
I20260809 12:22:34.516425 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:34.515957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:43517 (local address 127.0.0.1:37408)
0809 12:22:34.516166 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:34.516295 (+ 129us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 12:22:34.517436 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:34.517076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:43517 (local address 127.0.0.1:37412)
0809 12:22:34.517239 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:34.517314 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260809 12:22:34.517652 20930 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.253:43517: Network error: Client connection negotiation failed: client connection to 127.19.183.253:43517: connect: Connection refused (error 111)
I20260809 12:22:34.585952 20930 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } attempt: 2
I20260809 12:22:34.587303 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:34.586671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:43517 (local address 127.0.0.1:37418)
0809 12:22:34.586991 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:34.587112 (+ 121us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114}
I20260809 12:22:34.588428 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:34.587997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:43517 (local address 127.0.0.1:37420)
0809 12:22:34.588171 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:34.588262 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 12:22:34.588640 20930 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.253:43517: Network error: Client connection negotiation failed: client connection to 127.19.183.253:43517: connect: Connection refused (error 111)
I20260809 12:22:34.679945 20930 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } attempt: 3
I20260809 12:22:34.681360 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:34.680730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:43517 (local address 127.0.0.1:37434)
0809 12:22:34.681017 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:34.681156 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260809 12:22:34.682562 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:34.682069 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:43517 (local address 127.0.0.1:37446)
0809 12:22:34.682256 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:34.682355 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 12:22:34.682777 20930 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.253:43517: Network error: Client connection negotiation failed: client connection to 127.19.183.253:43517: connect: Connection refused (error 111)
I20260809 12:22:34.826051 20930 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } attempt: 4
I20260809 12:22:34.827482 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:34.826821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:43517 (local address 127.0.0.1:37458)
0809 12:22:34.827186 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:34.827301 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
I20260809 12:22:34.828613 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:34.828175 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:43517 (local address 127.0.0.1:37474)
0809 12:22:34.828343 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:34.828445 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:43517: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 12:22:34.828821 20930 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.253:43517: Network error: Client connection negotiation failed: client connection to 127.19.183.253:43517: connect: Connection refused (error 111)
W20260809 12:22:34.860137 20940 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 12:22:34.860436 20940 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:22:34.860538 20940 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:22:34.860700 20940 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:22:34.867865 20940 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:22:34.867944 20940 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:22:34.868077 20940 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:22:34.868155 20940 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:22:34.868266 20940 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:22:34.877801 20940 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:35929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:33983,127.19.183.253:43517,127.19.183.252:37575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.253.keytab
--principal=kudu/127.19.183.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.253:43517
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.183.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:22:34.879138 20940 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:22:34.880451 20940 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:22:34.892475 20951 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:34.892586 20954 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:34.892792 20949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:22:34.893528 20940 server_base.cc:1034] running on GCE node
I20260809 12:22:34.894272 20940 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:22:34.895659 20940 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:22:34.896907 20940 hybrid_clock.cc:648] HybridClock initialized: now 1786278154896845 us; error 56 us; skew 500 ppm
I20260809 12:22:34.897233 20940 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:22:34 dist-test-slave-39l8 krb5kdc[20819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278154, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:22:34.900368 20940 init.cc:377] Logged in from keytab as kudu/127.19.183.253@KRBTEST.COM (short username kudu)
I20260809 12:22:34.901746 20940 webserver.cc:460] Webserver started at http://127.19.183.253:37583/ using document root <none> and password file <none>
I20260809 12:22:34.902454 20940 fs_manager.cc:359] Metadata directory not provided
I20260809 12:22:34.902601 20940 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:22:34.902988 20940 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:22:34.905970 20940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-1/data/instance:
uuid: "4870714133be431f833aabdf22a99779"
format_stamp: "Formatted at 2026-08-09 12:22:34 on dist-test-slave-39l8"
server_key: "d095c9c3ad2aac75b378cec1811a44fe"
server_key_iv: "03fcda3eb307d19cb34c451fb3695f43"
server_key_version: "encryptionkey@0"
I20260809 12:22:34.906821 20940 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-1/wal/instance:
uuid: "4870714133be431f833aabdf22a99779"
format_stamp: "Formatted at 2026-08-09 12:22:34 on dist-test-slave-39l8"
server_key: "d095c9c3ad2aac75b378cec1811a44fe"
server_key_iv: "03fcda3eb307d19cb34c451fb3695f43"
server_key_version: "encryptionkey@0"
I20260809 12:22:34.911870 20940 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260809 12:22:34.915295 20962 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:22:34.916635 20940 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260809 12:22:34.916824 20940 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-1/wal
uuid: "4870714133be431f833aabdf22a99779"
format_stamp: "Formatted at 2026-08-09 12:22:34 on dist-test-slave-39l8"
server_key: "d095c9c3ad2aac75b378cec1811a44fe"
server_key_iv: "03fcda3eb307d19cb34c451fb3695f43"
server_key_version: "encryptionkey@0"
I20260809 12:22:34.917054 20940 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:22:34.950065 20940 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:22:34.954036 20940 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:22:34.954319 20940 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:22:34.973384 20940 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.253:43517
I20260809 12:22:34.973470 21030 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.253:43517 every 8 connection(s)
I20260809 12:22:34.974828 20940 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-1/data/info.pb
I20260809 12:22:34.978778 21031 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:22:34.983309 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 20940
I20260809 12:22:34.983505 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-1/wal/instance
I20260809 12:22:34.983944 20191 external_mini_cluster.cc:1428] Setting key fabfe3e98700865f9952e4ebab306ed4
I20260809 12:22:34.985666 21031 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33983 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:22:35 dist-test-slave-39l8 krb5kdc[20819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278154, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.253@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:22:35.014328 21036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:34.989437 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38592 (local address 127.19.183.254:33983)
0809 12:22:34.991309 (+ 1872us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:34.991318 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:34.995614 (+ 4296us) server_negotiation.cc:407] Connection header received
0809 12:22:34.997567 (+ 1953us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:34.997575 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:34.997674 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:34.997807 (+ 133us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:34.999441 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.000572 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.003119 (+ 2547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.003538 (+ 419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.003640 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:35.007979 (+ 4339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:35.007985 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:35.007988 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:35.008027 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:35.010943 (+ 2916us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.011939 (+ 996us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.011943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.011946 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.012081 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.012545 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.012549 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.012551 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.012827 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:35.012990 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:35.013696 (+ 706us) server_negotiation.cc:299] Negotiation successful
0809 12:22:35.013925 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1685,"thread_start_us":1494,"threads_started":1}
I20260809 12:22:35.015379 21039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:34.990037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:33983 (local address 127.0.0.1:38592)
0809 12:22:34.995348 (+ 5311us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:34.995445 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0809 12:22:34.997290 (+ 1845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:22:34.998007 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:22:34.998014 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:22:34.998569 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:22:34.999220 (+ 651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:34.999238 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.000770 (+ 1532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.000777 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:35.002960 (+ 2183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:35.002980 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.003776 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.003786 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:35.004109 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:35.004860 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:22:35.004894 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:22:35.007734 (+ 2840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:22:35.011097 (+ 3363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:35.011108 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:35.011124 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:35.011800 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:35.012210 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:35.012215 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:35.012219 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:35.012413 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:35.012966 (+ 553us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:22:35.012975 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:22:35.013333 (+ 358us) client_negotiation.cc:769] Sending connection context
0809 12:22:35.013606 (+ 273us) client_negotiation.cc:240] Negotiation successful
0809 12:22:35.014137 (+ 531us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5029,"thread_start_us":4825,"threads_started":1}
I20260809 12:22:35.017346 21031 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.19.183.252@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.252@KRBTEST.COM" created.
Aug 09 12:22:35 dist-test-slave-39l8 krb5kdc[20819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278154, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.253@KRBTEST.COM for kudu/127.19.183.253@KRBTEST.COM
I20260809 12:22:35.033702 21039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.018281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:43517 (local address 127.0.0.1:37486)
0809 12:22:35.018488 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.018520 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0809 12:22:35.018637 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:22:35.019474 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:22:35.019481 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:22:35.020044 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:22:35.020273 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:35.020287 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.021973 (+ 1686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.021977 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:35.022890 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:35.022903 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.023895 (+ 992us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.023903 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:35.024068 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:35.024665 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:22:35.024682 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:22:35.026697 (+ 2015us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:22:35.031290 (+ 4593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:35.031294 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:35.031296 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:35.031676 (+ 380us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:35.032128 (+ 452us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:35.032131 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:35.032133 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:35.032263 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:35.033000 (+ 737us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:22:35.033004 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:22:35.033131 (+ 127us) client_negotiation.cc:769] Sending connection context
0809 12:22:35.033245 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 12:22:35.033437 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 12:22:35.034221 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.018363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37486 (local address 127.19.183.253:43517)
0809 12:22:35.019042 (+ 679us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:35.019048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:35.019073 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:22:35.019128 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:35.019134 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:35.019292 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:35.019409 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:35.020505 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.021795 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.023026 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.023737 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.023836 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:35.026991 (+ 3155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:35.027007 (+ 16us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:35.027026 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:35.027069 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:35.031062 (+ 3993us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.031811 (+ 749us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.031823 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.031829 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.032008 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.032371 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.032378 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.032384 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.032876 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:35.033103 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:35.033408 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 12:22:35.033705 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":532,"thread_start_us":377,"threads_started":1}
I20260809 12:22:35.036679 21031 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:22:35.037727 21039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.037285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:37575 (local address 127.0.0.1:38240)
0809 12:22:35.037485 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.037583 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 12:22:35.037958 20966 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
I20260809 12:22:35.039538 21039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.039222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:37575 (local address 127.0.0.1:38250)
0809 12:22:35.039362 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.039442 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 12:22:35.039829 21031 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:37575: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.252.keytab.
Entry for principal kudu/127.19.183.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.252.keytab.
I20260809 12:22:35.063282 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.252:37575
--webserver_interface=127.19.183.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.252.keytab
--principal=kudu/127.19.183.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:35929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:33983,127.19.183.253:43517,127.19.183.252:37575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:22:35.089128 21031 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } attempt: 1
I20260809 12:22:35.090364 21039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.089868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:37575 (local address 127.0.0.1:38252)
0809 12:22:35.090065 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.090177 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 12:22:35.091560 21039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.091224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:37575 (local address 127.0.0.1:38268)
0809 12:22:35.091381 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.091457 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 12:22:35.091805 21031 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:37575: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
I20260809 12:22:35.128142 20930 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } attempt: 5
Aug 09 12:22:35 dist-test-slave-39l8 krb5kdc[20819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278154, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for kudu/127.19.183.253@KRBTEST.COM
I20260809 12:22:35.141659 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.129254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37500 (local address 127.19.183.253:43517)
0809 12:22:35.129466 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:35.129473 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:35.129497 (+ 24us) server_negotiation.cc:407] Connection header received
0809 12:22:35.129567 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:35.129571 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:35.129665 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:35.129775 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:35.130614 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.131487 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.132664 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.133106 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.133244 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:35.136493 (+ 3249us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:35.136498 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:35.136501 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:35.136531 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:35.139126 (+ 2595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.139774 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.139781 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.139784 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.139915 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.140295 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.140301 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.140304 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.140572 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:35.140724 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:35.141115 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 12:22:35.141337 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 12:22:35.141682 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.129019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:43517 (local address 127.0.0.1:37500)
0809 12:22:35.129225 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.129252 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0809 12:22:35.129401 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:22:35.129827 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:22:35.129833 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:22:35.130261 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:22:35.130454 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:35.130463 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.131631 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.131637 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:35.132499 (+ 862us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:35.132518 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.133268 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.133274 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:35.133478 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:35.134137 (+ 659us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:22:35.134155 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:22:35.136302 (+ 2147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:22:35.139264 (+ 2962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:35.139270 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:35.139274 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:35.139655 (+ 381us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:35.140031 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:35.140036 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:35.140040 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:35.140182 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:35.140702 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:22:35.140707 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:22:35.140859 (+ 152us) client_negotiation.cc:769] Sending connection context
0809 12:22:35.141004 (+ 145us) client_negotiation.cc:240] Negotiation successful
0809 12:22:35.141332 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 12:22:35.143023 20930 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:22:35.144152 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.143654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:37575 (local address 127.0.0.1:38276)
0809 12:22:35.143862 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.143966 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260809 12:22:35.145152 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.144757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:37575 (local address 127.0.0.1:38282)
0809 12:22:35.144924 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.145007 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 12:22:35.145370 20930 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:37575: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
I20260809 12:22:35.159997 21031 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } attempt: 2
I20260809 12:22:35.161023 21039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.160581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:37575 (local address 127.0.0.1:38294)
0809 12:22:35.160782 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.160874 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 12:22:35.162000 21039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.161687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:37575 (local address 127.0.0.1:38308)
0809 12:22:35.161838 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.161909 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 12:22:35.162249 21031 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:37575: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
I20260809 12:22:35.196612 20930 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } attempt: 1
I20260809 12:22:35.197868 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.197315 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:37575 (local address 127.0.0.1:38320)
0809 12:22:35.197543 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.197653 (+ 110us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260809 12:22:35.198907 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.198527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:37575 (local address 127.0.0.1:38326)
0809 12:22:35.198677 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.198784 (+ 107us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 12:22:35.199147 20930 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:37575: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
I20260809 12:22:35.253510 21031 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } attempt: 3
I20260809 12:22:35.254804 21039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.254250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:37575 (local address 127.0.0.1:38328)
0809 12:22:35.254457 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.254555 (+ 98us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
I20260809 12:22:35.255931 21039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.255502 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:37575 (local address 127.0.0.1:38334)
0809 12:22:35.255741 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.255825 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131}
W20260809 12:22:35.256191 21031 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:37575: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
I20260809 12:22:35.267397 20930 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } attempt: 2
I20260809 12:22:35.268760 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.268077 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:37575 (local address 127.0.0.1:38348)
0809 12:22:35.268383 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.268520 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260809 12:22:35.269841 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.269467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:37575 (local address 127.0.0.1:38356)
0809 12:22:35.269620 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.269704 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 12:22:35.270112 20930 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:37575: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
I20260809 12:22:35.376368 20930 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } attempt: 3
I20260809 12:22:35.377676 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.377158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:37575 (local address 127.0.0.1:38370)
0809 12:22:35.377404 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.377497 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 12:22:35.378816 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.378381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:37575 (local address 127.0.0.1:38382)
0809 12:22:35.378563 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.378650 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
W20260809 12:22:35.379062 20930 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:37575: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
I20260809 12:22:35.399430 21031 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } attempt: 4
I20260809 12:22:35.400605 21039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.400125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:37575 (local address 127.0.0.1:38394)
0809 12:22:35.400363 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.400449 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 12:22:35.401654 21039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.401311 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:37575 (local address 127.0.0.1:38404)
0809 12:22:35.401466 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.401542 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260809 12:22:35.401887 21031 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:37575: Network error: Client connection negotiation failed: client connection to 127.19.183.252:37575: connect: Connection refused (error 111)
W20260809 12:22:35.426533 21046 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 12:22:35.426944 21046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:22:35.427059 21046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:22:35.427202 21046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:22:35.434547 21046 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:22:35.434648 21046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:22:35.434752 21046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:22:35.434840 21046 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:22:35.434952 21046 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:22:35.446007 21046 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:35929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:33983,127.19.183.253:43517,127.19.183.252:37575
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.252.keytab
--principal=kudu/127.19.183.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.252:37575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.183.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:22:35.447576 21046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:22:35.449069 21046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:22:35.462283 21051 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:35.462359 21055 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:35.462220 21053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:22:35.463449 21046 server_base.cc:1034] running on GCE node
I20260809 12:22:35.464327 21046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:22:35.465806 21046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:22:35.467077 21046 hybrid_clock.cc:648] HybridClock initialized: now 1786278155467009 us; error 52 us; skew 500 ppm
I20260809 12:22:35.467434 21046 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:22:35 dist-test-slave-39l8 krb5kdc[20819](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278155, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:22:35.471053 21046 init.cc:377] Logged in from keytab as kudu/127.19.183.252@KRBTEST.COM (short username kudu)
I20260809 12:22:35.472522 21046 webserver.cc:460] Webserver started at http://127.19.183.252:43209/ using document root <none> and password file <none>
I20260809 12:22:35.473282 21046 fs_manager.cc:359] Metadata directory not provided
I20260809 12:22:35.473433 21046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:22:35.473791 21046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:22:35.477020 21046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-2/data/instance:
uuid: "b32952f2eeca447eab9ee78f68c7e5d9"
format_stamp: "Formatted at 2026-08-09 12:22:35 on dist-test-slave-39l8"
server_key: "d5f9473b41cdb6fbb5710994ca4fd7db"
server_key_iv: "bb233ceadbc2f7c91d5e9fccf1e08aa4"
server_key_version: "encryptionkey@0"
I20260809 12:22:35.477900 21046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-2/wal/instance:
uuid: "b32952f2eeca447eab9ee78f68c7e5d9"
format_stamp: "Formatted at 2026-08-09 12:22:35 on dist-test-slave-39l8"
server_key: "d5f9473b41cdb6fbb5710994ca4fd7db"
server_key_iv: "bb233ceadbc2f7c91d5e9fccf1e08aa4"
server_key_version: "encryptionkey@0"
I20260809 12:22:35.483335 21046 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260809 12:22:35.487178 21066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:22:35.488617 21046 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260809 12:22:35.488770 21046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-2/wal
uuid: "b32952f2eeca447eab9ee78f68c7e5d9"
format_stamp: "Formatted at 2026-08-09 12:22:35 on dist-test-slave-39l8"
server_key: "d5f9473b41cdb6fbb5710994ca4fd7db"
server_key_iv: "bb233ceadbc2f7c91d5e9fccf1e08aa4"
server_key_version: "encryptionkey@0"
I20260809 12:22:35.488965 21046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:22:35.524542 21046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:22:35.529407 21046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:22:35.529663 21046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:22:35.550375 21046 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.252:37575
I20260809 12:22:35.550392 21149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.252:37575 every 8 connection(s)
I20260809 12:22:35.551882 21046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-2/data/info.pb
I20260809 12:22:35.555692 21151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:22:35.556298 20930 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } attempt: 4
I20260809 12:22:35.558533 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 21046
I20260809 12:22:35.558748 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786277774779245-20191-0/minicluster-data/master-2/wal/instance
I20260809 12:22:35.559293 20191 external_mini_cluster.cc:1428] Setting key ffd36d116be79cd19f5b23bee065fdf1
I20260809 12:22:35.567464 21151 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33983 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:22:35 dist-test-slave-39l8 krb5kdc[20819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278154, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for kudu/127.19.183.252@KRBTEST.COM
Aug 09 12:22:35 dist-test-slave-39l8 krb5kdc[20819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.252@KRBTEST.COM
I20260809 12:22:35.581171 20932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.557024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:37575 (local address 127.0.0.1:38420)
0809 12:22:35.557269 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.557289 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:22:35.557449 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:22:35.562420 (+ 4971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:22:35.562425 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:22:35.563070 (+ 645us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:22:35.563312 (+ 242us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:35.563330 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.565798 (+ 2468us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.565804 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:35.567403 (+ 1599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:35.567418 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.569158 (+ 1740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.569165 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:35.569390 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:35.570502 (+ 1112us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:22:35.570547 (+ 45us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:22:35.573197 (+ 2650us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:22:35.577593 (+ 4396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:35.577599 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:35.577604 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:35.578270 (+ 666us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:35.578871 (+ 601us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:35.578879 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:35.578883 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:35.579052 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:35.580085 (+ 1033us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:22:35.580090 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:22:35.580299 (+ 209us) client_negotiation.cc:769] Sending connection context
0809 12:22:35.580451 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 12:22:35.580773 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":82}
Aug 09 12:22:35 dist-test-slave-39l8 krb5kdc[20819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278155, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.252@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:22:35.582844 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.560312 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38432 (local address 127.19.183.252:37575)
0809 12:22:35.561363 (+ 1051us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:35.561367 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:35.561388 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:22:35.561604 (+ 216us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:35.561617 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:35.562067 (+ 450us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:35.562424 (+ 357us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:35.564479 (+ 2055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.565942 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.567449 (+ 1507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.568947 (+ 1498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.569154 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:35.574936 (+ 5782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:35.574942 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:35.574945 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:35.575008 (+ 63us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:35.579366 (+ 4358us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.580215 (+ 849us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.580220 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.580224 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.580361 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.580837 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.580844 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.580848 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.581216 (+ 368us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:35.581497 (+ 281us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:35.582046 (+ 549us) server_negotiation.cc:299] Negotiation successful
0809 12:22:35.582265 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":769,"server-negotiator.queue_time_us":949,"spinlock_wait_cycles":11264,"thread_start_us":793,"threads_started":1}
I20260809 12:22:35.582866 21153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.558013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38420 (local address 127.19.183.252:37575)
0809 12:22:35.558971 (+ 958us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:35.558982 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:35.559032 (+ 50us) server_negotiation.cc:407] Connection header received
0809 12:22:35.561590 (+ 2558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:35.561613 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:35.562062 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:35.562423 (+ 361us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:35.563521 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.565538 (+ 2017us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.567580 (+ 2042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.568947 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.569731 (+ 784us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:35.573450 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:35.573461 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:35.573471 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:35.573507 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:35.577452 (+ 3945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.578412 (+ 960us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.578422 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.578430 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.578674 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.579213 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.579225 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.579229 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.579912 (+ 683us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:35.580168 (+ 256us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:35.580745 (+ 577us) server_negotiation.cc:299] Negotiation successful
0809 12:22:35.581221 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":100,"server-negotiator.queue_time_us":729,"spinlock_wait_cycles":68352,"thread_start_us":449,"threads_started":1}
I20260809 12:22:35.589972 21157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.570100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:33983 (local address 127.0.0.1:38598)
0809 12:22:35.571401 (+ 1301us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.571467 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 12:22:35.571664 (+ 197us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:22:35.572219 (+ 555us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:22:35.572229 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:22:35.573181 (+ 952us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:22:35.573647 (+ 466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:35.573668 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.575112 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.575125 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:35.577316 (+ 2191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:35.577331 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.578235 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.578241 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:35.578463 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:35.579232 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:22:35.579256 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:22:35.582209 (+ 2953us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:22:35.587132 (+ 4923us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:35.587146 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:35.587155 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:35.587730 (+ 575us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:35.588140 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:35.588145 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:35.588149 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:35.588290 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:35.588865 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:22:35.588873 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:22:35.589034 (+ 161us) client_negotiation.cc:769] Sending connection context
0809 12:22:35.589241 (+ 207us) client_negotiation.cc:240] Negotiation successful
0809 12:22:35.589551 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1047,"thread_start_us":908,"threads_started":1}
I20260809 12:22:35.590049 21156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.570101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38598 (local address 127.19.183.254:33983)
0809 12:22:35.570810 (+ 709us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:35.570817 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:35.571755 (+ 938us) server_negotiation.cc:407] Connection header received
0809 12:22:35.571823 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:35.571830 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:35.571918 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:35.572094 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:35.573824 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.574893 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.577549 (+ 2656us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.578021 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.578171 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:35.582426 (+ 4255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:35.582434 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:35.582438 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:35.582478 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:35.586935 (+ 4457us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.587870 (+ 935us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.587875 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.587878 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.588039 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.588410 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.588415 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.588419 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.588721 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:35.588878 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:35.589335 (+ 457us) server_negotiation.cc:299] Negotiation successful
0809 12:22:35.589562 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":1538,"threads_started":1}
I20260809 12:22:35.591176 20930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6: Bootstrap starting.
I20260809 12:22:35.591799 21151 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:22:35.595723 20930 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6: Neither blocks nor log segments found. Creating new log.
I20260809 12:22:35.597088 20930 log.cc:824] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6: Log is configured to *not* fsync() on all Append() calls
Aug 09 12:22:35 dist-test-slave-39l8 krb5kdc[20819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:22:35.600703 20930 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6: No bootstrap required, opened a new log
Aug 09 12:22:35 dist-test-slave-39l8 krb5kdc[20819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278155, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.252@KRBTEST.COM for kudu/127.19.183.253@KRBTEST.COM
I20260809 12:22:35.606359 21159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.586660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38606 (local address 127.19.183.254:33983)
0809 12:22:35.587418 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:35.587424 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:35.587447 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:22:35.587522 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:35.587530 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:35.587604 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:35.587727 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:35.589503 (+ 1776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.591006 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.593034 (+ 2028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.593856 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.594008 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:35.599042 (+ 5034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:35.599048 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:35.599051 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:35.599085 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:35.602547 (+ 3462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.603564 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.603569 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.603573 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.603744 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.604251 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.604258 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.604262 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.604691 (+ 429us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:35.604977 (+ 286us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:35.605668 (+ 691us) server_negotiation.cc:299] Negotiation successful
0809 12:22:35.605902 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":624,"thread_start_us":367,"threads_started":1}
I20260809 12:22:35.609906 21157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.592649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:43517 (local address 127.0.0.1:37502)
0809 12:22:35.592998 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.593013 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 12:22:35.593142 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:22:35.593471 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:22:35.593476 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:22:35.594032 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:22:35.594214 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:35.594225 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.596080 (+ 1855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.596086 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:35.598698 (+ 2612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:35.598749 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.599814 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.599823 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:35.600773 (+ 950us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:35.601353 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:22:35.601370 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:22:35.603285 (+ 1915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:22:35.607237 (+ 3952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:35.607242 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:35.607246 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:35.607749 (+ 503us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:35.608199 (+ 450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:35.608204 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:35.608208 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:35.608382 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:35.608938 (+ 556us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:22:35.608943 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:22:35.609100 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 12:22:35.609243 (+ 143us) client_negotiation.cc:240] Negotiation successful
0809 12:22:35.609518 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202}
I20260809 12:22:35.609956 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.592610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37502 (local address 127.19.183.253:43517)
0809 12:22:35.592826 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:35.592832 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:35.593096 (+ 264us) server_negotiation.cc:407] Connection header received
0809 12:22:35.593264 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:35.593270 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:35.593344 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:35.593449 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:35.594395 (+ 946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.595884 (+ 1489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.598902 (+ 3018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.599593 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.599744 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:35.603484 (+ 3740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:35.603514 (+ 30us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:35.603521 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:35.603565 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:35.607106 (+ 3541us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.607868 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.607874 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.607877 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.608053 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.608515 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.608527 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.608531 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.608800 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:35.608968 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:35.609338 (+ 370us) server_negotiation.cc:299] Negotiation successful
0809 12:22:35.609547 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260809 12:22:35.611503 21151 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:22:35.614147 20930 raft_consensus.cc:348] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08f78843868434f9b019cf28c5039f6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33983 } } peers { permanent_uuid: "4870714133be431f833aabdf22a99779" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } } peers { permanent_uuid: "b32952f2eeca447eab9ee78f68c7e5d9" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } }
I20260809 12:22:35.614530 20930 raft_consensus.cc:729] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f08f78843868434f9b019cf28c5039f6, State: Initialized, Role: FOLLOWER
I20260809 12:22:35.615170 20930 consensus_queue.cc:260] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08f78843868434f9b019cf28c5039f6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33983 } } peers { permanent_uuid: "4870714133be431f833aabdf22a99779" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } } peers { permanent_uuid: "b32952f2eeca447eab9ee78f68c7e5d9" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } }
I20260809 12:22:35.616243 21164 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08f78843868434f9b019cf28c5039f6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33983 } } peers { permanent_uuid: "4870714133be431f833aabdf22a99779" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } } peers { permanent_uuid: "b32952f2eeca447eab9ee78f68c7e5d9" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } } }
I20260809 12:22:35.616739 21164 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:22:35.617774 20930 sys_catalog.cc:564] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6 [sys.catalog]: configured and running, proceeding with master startup.
Aug 09 12:22:35 dist-test-slave-39l8 krb5kdc[20819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278155, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.252@KRBTEST.COM for kudu/127.19.183.252@KRBTEST.COM
Aug 09 12:22:35 dist-test-slave-39l8 krb5kdc[20819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.253@KRBTEST.COM
I20260809 12:22:35.627832 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.612406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38440 (local address 127.19.183.252:37575)
0809 12:22:35.612584 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:35.612591 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:35.612606 (+ 15us) server_negotiation.cc:407] Connection header received
0809 12:22:35.612649 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:35.612655 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:35.612722 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:35.612805 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:35.613492 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.614592 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.616534 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.617237 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.617348 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:35.622051 (+ 4703us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:35.622058 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:35.622064 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:35.622106 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:35.624904 (+ 2798us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.625663 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.625670 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.625673 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.625793 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.626232 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.626236 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.626239 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.626560 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:35.626647 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:35.627263 (+ 616us) server_negotiation.cc:299] Negotiation successful
0809 12:22:35.627488 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 12:22:35.628690 21157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.612181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:37575 (local address 127.0.0.1:38440)
0809 12:22:35.612332 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.612350 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 12:22:35.612441 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:22:35.612862 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:22:35.612866 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:22:35.613220 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:22:35.613354 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:35.613362 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.614812 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.614818 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:35.616365 (+ 1547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:35.616382 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.617583 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.617589 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:35.617822 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:35.618474 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:22:35.618499 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:22:35.621817 (+ 3318us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:22:35.625047 (+ 3230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:35.625052 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:35.625056 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:35.625551 (+ 495us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:35.625900 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:35.625905 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:35.625908 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:35.626104 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:35.626771 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:22:35.626777 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:22:35.626971 (+ 194us) client_negotiation.cc:769] Sending connection context
0809 12:22:35.628135 (+ 1164us) client_negotiation.cc:240] Negotiation successful
0809 12:22:35.628417 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 12:22:35.631520 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.608322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37512 (local address 127.19.183.253:43517)
0809 12:22:35.610208 (+ 1886us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:35.610216 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:35.611255 (+ 1039us) server_negotiation.cc:407] Connection header received
0809 12:22:35.611521 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:35.611527 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:35.611608 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:35.611685 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:35.613647 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.614688 (+ 1041us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.616230 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.616880 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.616975 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:35.622520 (+ 5545us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:35.622530 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:35.622538 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:35.622596 (+ 58us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:35.627444 (+ 4848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.628280 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.628290 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.628297 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.628572 (+ 275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.629119 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.629128 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.629134 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.629606 (+ 472us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:35.629980 (+ 374us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:35.630924 (+ 944us) server_negotiation.cc:299] Negotiation successful
0809 12:22:35.631152 (+ 228us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1768,"thread_start_us":2536,"threads_started":1}
I20260809 12:22:35.634003 21151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32952f2eeca447eab9ee78f68c7e5d9: Bootstrap starting.
I20260809 12:22:35.637244 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 12:22:35.638383 21151 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b32952f2eeca447eab9ee78f68c7e5d9: Neither blocks nor log segments found. Creating new log.
I20260809 12:22:35.640106 21151 log.cc:824] T 00000000000000000000000000000000 P b32952f2eeca447eab9ee78f68c7e5d9: Log is configured to *not* fsync() on all Append() calls
I20260809 12:22:35.640799 20895 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "f08f78843868434f9b019cf28c5039f6"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38606
I20260809 12:22:35.641319 20895 raft_consensus.cc:482] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 12:22:35.641532 20895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6 [term 0 FOLLOWER]: Advancing to term 1
W20260809 12:22:35.644023 21184 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 12:22:35.644199 21184 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 12:22:35.645356 20895 raft_consensus.cc:504] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08f78843868434f9b019cf28c5039f6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33983 } } peers { permanent_uuid: "4870714133be431f833aabdf22a99779" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } } peers { permanent_uuid: "b32952f2eeca447eab9ee78f68c7e5d9" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } }
I20260809 12:22:35.646818 21151 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b32952f2eeca447eab9ee78f68c7e5d9: No bootstrap required, opened a new log
I20260809 12:22:35.646939 20895 leader_election.cc:288] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6 [CANDIDATE]: Term 1 election: Requested vote from peers 4870714133be431f833aabdf22a99779 (127.19.183.253:43517), b32952f2eeca447eab9ee78f68c7e5d9 (127.19.183.252:37575)
W20260809 12:22:35.656078 20861 tablet.cc:2369] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:22:35.645546 20834 master_runner.cc:186] Error getting master registration for 127.19.183.253:43517: OK, instance_id { permanent_uuid: "4870714133be431f833aabdf22a99779" instance_seqno: 1786278154968024 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 12:22:35.660466 20834 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 12:22:35.663349 21151 raft_consensus.cc:348] T 00000000000000000000000000000000 P b32952f2eeca447eab9ee78f68c7e5d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08f78843868434f9b019cf28c5039f6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33983 } } peers { permanent_uuid: "4870714133be431f833aabdf22a99779" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } } peers { permanent_uuid: "b32952f2eeca447eab9ee78f68c7e5d9" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } }
I20260809 12:22:35.663836 21151 raft_consensus.cc:729] T 00000000000000000000000000000000 P b32952f2eeca447eab9ee78f68c7e5d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b32952f2eeca447eab9ee78f68c7e5d9, State: Initialized, Role: FOLLOWER
I20260809 12:22:35.664352 21151 consensus_queue.cc:260] T 00000000000000000000000000000000 P b32952f2eeca447eab9ee78f68c7e5d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08f78843868434f9b019cf28c5039f6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33983 } } peers { permanent_uuid: "4870714133be431f833aabdf22a99779" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } } peers { permanent_uuid: "b32952f2eeca447eab9ee78f68c7e5d9" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } }
I20260809 12:22:35.666978 21151 sys_catalog.cc:564] T 00000000000000000000000000000000 P b32952f2eeca447eab9ee78f68c7e5d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:22:35.667064 21106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f08f78843868434f9b019cf28c5039f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b32952f2eeca447eab9ee78f68c7e5d9"
I20260809 12:22:35.667673 21106 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b32952f2eeca447eab9ee78f68c7e5d9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:22:35.671648 21106 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b32952f2eeca447eab9ee78f68c7e5d9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f08f78843868434f9b019cf28c5039f6 in term 1.
I20260809 12:22:35.672575 20858 leader_election.cc:302] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b32952f2eeca447eab9ee78f68c7e5d9, f08f78843868434f9b019cf28c5039f6; no voters:
I20260809 12:22:35.673134 21164 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:22:35.666157 21195 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32952f2eeca447eab9ee78f68c7e5d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08f78843868434f9b019cf28c5039f6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33983 } } peers { permanent_uuid: "4870714133be431f833aabdf22a99779" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } } peers { permanent_uuid: "b32952f2eeca447eab9ee78f68c7e5d9" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } } }
I20260809 12:22:35.673980 21195 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32952f2eeca447eab9ee78f68c7e5d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:22:35.688617 21164 raft_consensus.cc:686] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6 [term 1 LEADER]: Becoming Leader. State: Replica: f08f78843868434f9b019cf28c5039f6, State: Running, Role: LEADER
I20260809 12:22:35.689299 21164 consensus_queue.cc:237] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08f78843868434f9b019cf28c5039f6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33983 } } peers { permanent_uuid: "4870714133be431f833aabdf22a99779" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } } peers { permanent_uuid: "b32952f2eeca447eab9ee78f68c7e5d9" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } }
I20260809 12:22:35.701108 21031 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } attempt: 5
I20260809 12:22:35.702900 21200 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f08f78843868434f9b019cf28c5039f6. Latest consensus state: current_term: 1 leader_uuid: "f08f78843868434f9b019cf28c5039f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08f78843868434f9b019cf28c5039f6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33983 } } peers { permanent_uuid: "4870714133be431f833aabdf22a99779" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } } peers { permanent_uuid: "b32952f2eeca447eab9ee78f68c7e5d9" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } } }
I20260809 12:22:35.703372 21200 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6 [sys.catalog]: This master's current role is: LEADER
Aug 09 12:22:35 dist-test-slave-39l8 krb5kdc[20819](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278154, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.253@KRBTEST.COM for kudu/127.19.183.252@KRBTEST.COM
I20260809 12:22:35.713614 21211 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 12:22:35.716622 21211 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:22:35.719722 21039 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.701686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:37575 (local address 127.0.0.1:38450)
0809 12:22:35.702058 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:35.702079 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0809 12:22:35.702198 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:22:35.703180 (+ 982us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:22:35.703186 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:22:35.703745 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:22:35.703966 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:35.703978 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.705444 (+ 1466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.705450 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:35.706939 (+ 1489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:35.706954 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.708127 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.708134 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:35.708387 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:35.709090 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:22:35.709117 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:22:35.712176 (+ 3059us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:22:35.716436 (+ 4260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:35.716444 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:35.716449 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:35.717001 (+ 552us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:35.717487 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:35.717494 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:35.717499 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:35.717695 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:35.718509 (+ 814us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:22:35.718515 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:22:35.718759 (+ 244us) client_negotiation.cc:769] Sending connection context
0809 12:22:35.718937 (+ 178us) client_negotiation.cc:240] Negotiation successful
0809 12:22:35.719281 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245}
I20260809 12:22:35.720796 21153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.702432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38450 (local address 127.19.183.252:37575)
0809 12:22:35.702654 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:35.702661 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:35.702684 (+ 23us) server_negotiation.cc:407] Connection header received
0809 12:22:35.702765 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:35.702772 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:35.702867 (+ 95us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:35.702973 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:35.704159 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.705282 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.707117 (+ 1835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.707775 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.707903 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:35.712392 (+ 4489us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:35.712399 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:35.712406 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:35.712451 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:35.716280 (+ 3829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.717161 (+ 881us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.717166 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.717169 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.717341 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.717845 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.717850 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.717854 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.718271 (+ 417us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:35.718428 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:35.720135 (+ 1707us) server_negotiation.cc:299] Negotiation successful
0809 12:22:35.720427 (+ 292us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 12:22:35.727478 21031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4870714133be431f833aabdf22a99779: Bootstrap starting.
I20260809 12:22:35.732223 21031 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4870714133be431f833aabdf22a99779: Neither blocks nor log segments found. Creating new log.
I20260809 12:22:35.733683 21031 log.cc:824] T 00000000000000000000000000000000 P 4870714133be431f833aabdf22a99779: Log is configured to *not* fsync() on all Append() calls
I20260809 12:22:35.735630 21159 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.671357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38618 (local address 127.19.183.254:33983)
0809 12:22:35.671579 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:35.671586 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:35.682217 (+ 10631us) server_negotiation.cc:407] Connection header received
0809 12:22:35.682884 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:35.682893 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:35.682997 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:35.683089 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:35.686118 (+ 3029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.687548 (+ 1430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.690460 (+ 2912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.691289 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.691392 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:35.695472 (+ 4080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:35.695481 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:35.695489 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:35.695541 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:35.700090 (+ 4549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.701011 (+ 921us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.701017 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.701021 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.701175 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.701626 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.701632 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.701636 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.702082 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:35.702375 (+ 293us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:35.734946 (+ 32571us) server_negotiation.cc:299] Negotiation successful
0809 12:22:35.735236 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 12:22:35.738023 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.678153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37528 (local address 127.19.183.253:43517)
0809 12:22:35.678386 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:35.678395 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:35.692885 (+ 14490us) server_negotiation.cc:407] Connection header received
0809 12:22:35.693333 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:35.693340 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:35.693437 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:35.693521 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:35.700541 (+ 7020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.701685 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.709815 (+ 8130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.710610 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.710734 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:35.725280 (+ 14546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:35.725286 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:35.725290 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:35.725328 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:35.729578 (+ 4250us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.733093 (+ 3515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.733099 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.733104 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.733275 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.734807 (+ 1532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.734812 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.734815 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.735198 (+ 383us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:35.735437 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:35.737418 (+ 1981us) server_negotiation.cc:299] Negotiation successful
0809 12:22:35.737686 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 12:22:35.741204 21031 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4870714133be431f833aabdf22a99779: No bootstrap required, opened a new log
I20260809 12:22:35.748755 21106 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b32952f2eeca447eab9ee78f68c7e5d9 [term 1 FOLLOWER]: Refusing update from remote peer f08f78843868434f9b019cf28c5039f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:22:35.750237 21164 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b32952f2eeca447eab9ee78f68c7e5d9" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:22:35.752207 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:35.691645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38446 (local address 127.19.183.252:37575)
0809 12:22:35.691829 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:35.691835 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:35.705237 (+ 13402us) server_negotiation.cc:407] Connection header received
0809 12:22:35.705928 (+ 691us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:35.705934 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:35.706030 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:35.706124 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:35.714821 (+ 8697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.716008 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.723159 (+ 7151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:35.723837 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:35.723936 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:35.742990 (+ 19054us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:35.742997 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:35.743002 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:35.743055 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:35.747014 (+ 3959us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.748589 (+ 1575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.748596 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.748599 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.748778 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:35.749733 (+ 955us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:35.749738 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:35.749742 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:35.750157 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:35.750403 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:35.751512 (+ 1109us) server_negotiation.cc:299] Negotiation successful
0809 12:22:35.751794 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 12:22:35.753456 21031 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4870714133be431f833aabdf22a99779 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08f78843868434f9b019cf28c5039f6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33983 } } peers { permanent_uuid: "4870714133be431f833aabdf22a99779" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } } peers { permanent_uuid: "b32952f2eeca447eab9ee78f68c7e5d9" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } }
I20260809 12:22:35.753867 21031 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4870714133be431f833aabdf22a99779 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4870714133be431f833aabdf22a99779, State: Initialized, Role: FOLLOWER
I20260809 12:22:35.754429 21031 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4870714133be431f833aabdf22a99779 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08f78843868434f9b019cf28c5039f6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33983 } } peers { permanent_uuid: "4870714133be431f833aabdf22a99779" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } } peers { permanent_uuid: "b32952f2eeca447eab9ee78f68c7e5d9" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } }
I20260809 12:22:35.761401 21227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4870714133be431f833aabdf22a99779 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08f78843868434f9b019cf28c5039f6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33983 } } peers { permanent_uuid: "4870714133be431f833aabdf22a99779" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } } peers { permanent_uuid: "b32952f2eeca447eab9ee78f68c7e5d9" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } } }
I20260809 12:22:35.761854 21227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4870714133be431f833aabdf22a99779 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:22:35.763782 21031 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4870714133be431f833aabdf22a99779 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:22:35.764114 20998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4870714133be431f833aabdf22a99779 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:22:35.767040 20999 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "f08f78843868434f9b019cf28c5039f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4870714133be431f833aabdf22a99779"
I20260809 12:22:35.773900 20998 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4870714133be431f833aabdf22a99779 [term 1 FOLLOWER]: Refusing update from remote peer f08f78843868434f9b019cf28c5039f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260809 12:22:35.781832 21074 tablet.cc:2369] T 00000000000000000000000000000000 P b32952f2eeca447eab9ee78f68c7e5d9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:22:35.782280 20999 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4870714133be431f833aabdf22a99779 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f08f78843868434f9b019cf28c5039f6 in term 1.
I20260809 12:22:35.784390 21164 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4870714133be431f833aabdf22a99779" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:22:35.792131 21195 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32952f2eeca447eab9ee78f68c7e5d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f08f78843868434f9b019cf28c5039f6. Latest consensus state: current_term: 1 leader_uuid: "f08f78843868434f9b019cf28c5039f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08f78843868434f9b019cf28c5039f6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33983 } } peers { permanent_uuid: "4870714133be431f833aabdf22a99779" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } } peers { permanent_uuid: "b32952f2eeca447eab9ee78f68c7e5d9" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } } }
I20260809 12:22:35.792567 21195 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32952f2eeca447eab9ee78f68c7e5d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:22:35.809000 21227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4870714133be431f833aabdf22a99779 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f08f78843868434f9b019cf28c5039f6. Latest consensus state: current_term: 1 leader_uuid: "f08f78843868434f9b019cf28c5039f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08f78843868434f9b019cf28c5039f6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33983 } } peers { permanent_uuid: "4870714133be431f833aabdf22a99779" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } } peers { permanent_uuid: "b32952f2eeca447eab9ee78f68c7e5d9" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } } }
I20260809 12:22:35.809369 21227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4870714133be431f833aabdf22a99779 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:22:35.810946 21195 sys_catalog.cc:455] T 00000000000000000000000000000000 P b32952f2eeca447eab9ee78f68c7e5d9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f08f78843868434f9b019cf28c5039f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08f78843868434f9b019cf28c5039f6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33983 } } peers { permanent_uuid: "4870714133be431f833aabdf22a99779" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } } peers { permanent_uuid: "b32952f2eeca447eab9ee78f68c7e5d9" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } } }
I20260809 12:22:35.811211 21195 sys_catalog.cc:458] T 00000000000000000000000000000000 P b32952f2eeca447eab9ee78f68c7e5d9 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:22:35.811259 21200 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f08f78843868434f9b019cf28c5039f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08f78843868434f9b019cf28c5039f6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33983 } } peers { permanent_uuid: "4870714133be431f833aabdf22a99779" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } } peers { permanent_uuid: "b32952f2eeca447eab9ee78f68c7e5d9" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } } }
I20260809 12:22:35.811628 21200 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:35.812547 21211 catalog_manager.cc:1347] Generated new cluster ID: 652aac25017a4ff9badca64ff140847e
I20260809 12:22:35.812668 21211 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:22:35.825443 21245 catalog_manager.cc:1259] Loaded cluster ID: 652aac25017a4ff9badca64ff140847e
I20260809 12:22:35.825618 21245 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b32952f2eeca447eab9ee78f68c7e5d9: loading cluster ID for follower catalog manager: success
W20260809 12:22:35.829567 21245 catalog_manager.cc:1576] T 00000000000000000000000000000000 P b32952f2eeca447eab9ee78f68c7e5d9: acquiring CA information for follower catalog manager: Not found: root CA entry not found
W20260809 12:22:35.829702 21245 catalog_manager.cc:873] Not found: root CA entry not found: failed to prepare follower catalog manager, will retry
I20260809 12:22:35.830940 21259 sys_catalog.cc:455] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f08f78843868434f9b019cf28c5039f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08f78843868434f9b019cf28c5039f6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33983 } } peers { permanent_uuid: "4870714133be431f833aabdf22a99779" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } } peers { permanent_uuid: "b32952f2eeca447eab9ee78f68c7e5d9" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } } }
I20260809 12:22:35.832485 21259 sys_catalog.cc:458] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:35.840363 21227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4870714133be431f833aabdf22a99779 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f08f78843868434f9b019cf28c5039f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f08f78843868434f9b019cf28c5039f6" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 33983 } } peers { permanent_uuid: "4870714133be431f833aabdf22a99779" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 43517 } } peers { permanent_uuid: "b32952f2eeca447eab9ee78f68c7e5d9" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 37575 } } }
I20260809 12:22:35.840638 21227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4870714133be431f833aabdf22a99779 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:22:35.875487 21211 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:22:35.878185 21211 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:22:35.816805 21046 master_runner.cc:186] Error getting master registration for 127.19.183.253:43517: OK, instance_id { permanent_uuid: "4870714133be431f833aabdf22a99779" instance_seqno: 1786278154968024 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 12:22:35.879034 21046 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 12:22:35.924855 21211 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f08f78843868434f9b019cf28c5039f6: Generated new TSK 0
I20260809 12:22:35.928252 21211 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:22:35.976670 21268 catalog_manager.cc:1259] Loaded cluster ID: 652aac25017a4ff9badca64ff140847e
I20260809 12:22:35.976904 21268 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4870714133be431f833aabdf22a99779: loading cluster ID for follower catalog manager: success
I20260809 12:22:35.993129 21268 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4870714133be431f833aabdf22a99779: acquiring CA information for follower catalog manager: success
I20260809 12:22:36.002928 21268 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4870714133be431f833aabdf22a99779: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 12:22:36.088129 20191 mini_kdc.cc:348] Time spent kdestroy: real 0.107s user 0.000s sys 0.033s
W20260809 12:22:36.142784 21159 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:36.129345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38632 (local address 127.19.183.254:33983)
0809 12:22:36.129538 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:36.129544 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:36.140560 (+ 11016us) server_negotiation.cc:407] Connection header received
0809 12:22:36.141062 (+ 502us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:36.141069 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:36.141188 (+ 119us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:36.141270 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:36.142476 (+ 1206us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38632: BlockingRecv error: recv got EOF from 127.0.0.1:38632 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 12:22:36.145417 21042 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:36.136327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37544 (local address 127.19.183.253:43517)
0809 12:22:36.136626 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:36.136635 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:36.143796 (+ 7161us) server_negotiation.cc:407] Connection header received
0809 12:22:36.144056 (+ 260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:36.144063 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:36.144145 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:36.144216 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:36.145162 (+ 946us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37544: BlockingRecv error: recv got EOF from 127.0.0.1:37544 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260809 12:22:36.162171 21155 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:36.159539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38460 (local address 127.19.183.252:37575)
0809 12:22:36.159732 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:36.159739 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:36.160376 (+ 637us) server_negotiation.cc:407] Connection header received
0809 12:22:36.160451 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:36.160456 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:36.160576 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:36.160673 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:36.161886 (+ 1213us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38460: BlockingRecv error: recv got EOF from 127.0.0.1:38460 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260809 12:22:36.214310 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:36.197775 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37558 (local address 127.19.183.253:43517)
0809 12:22:36.197985 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:36.197991 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:36.205740 (+ 7749us) server_negotiation.cc:407] Connection header received
0809 12:22:36.206338 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:36.206345 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:36.206606 (+ 261us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:36.206805 (+ 199us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:22:36.207573 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:36.208822 (+ 1249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:36.211022 (+ 2200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:36.211747 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:36.211858 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:36.212693 (+ 835us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:22:36.213408 (+ 715us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:22:36.213560 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:36.213838 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 12:22:36.213941 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
W20260809 12:22:36.266390 21155 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:36.252750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38464 (local address 127.19.183.252:37575)
0809 12:22:36.252953 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:36.252963 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:36.264016 (+ 11053us) server_negotiation.cc:407] Connection header received
0809 12:22:36.264614 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:36.264623 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:36.264774 (+ 151us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:36.264876 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:36.266118 (+ 1242us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38464: BlockingRecv error: recv got EOF from 127.0.0.1:38464 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
I20260809 12:22:36.317324 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:36.302209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37568 (local address 127.19.183.253:43517)
0809 12:22:36.302460 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:36.302466 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:36.309927 (+ 7461us) server_negotiation.cc:407] Connection header received
0809 12:22:36.310463 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:36.310470 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:36.310584 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:36.310664 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:22:36.311437 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:36.312660 (+ 1223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:36.314775 (+ 2115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:36.315536 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:36.315654 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:36.316337 (+ 683us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:22:36.316530 (+ 193us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:22:36.316633 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:36.316848 (+ 215us) server_negotiation.cc:299] Negotiation successful
0809 12:22:36.316956 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
W20260809 12:22:36.333658 21155 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:36.331172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38470 (local address 127.19.183.252:37575)
0809 12:22:36.331357 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:36.331364 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:36.331948 (+ 584us) server_negotiation.cc:407] Connection header received
0809 12:22:36.332177 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:36.332184 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:36.332312 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:36.332430 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:36.333383 (+ 953us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38470: BlockingRecv error: recv got EOF from 127.0.0.1:38470 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
I20260809 12:22:36.347585 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:36.340684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37576 (local address 127.19.183.253:43517)
0809 12:22:36.340876 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:36.340882 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:36.341418 (+ 536us) server_negotiation.cc:407] Connection header received
0809 12:22:36.341611 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:36.341617 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:36.341699 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:36.341804 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:22:36.342290 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:36.343457 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:36.345446 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:36.346097 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:36.346218 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:36.346769 (+ 551us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:22:36.346908 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:22:36.346996 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:36.347190 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 12:22:36.347270 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 12:22:36.356531 21155 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:36.354483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38478 (local address 127.19.183.252:37575)
0809 12:22:36.354681 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:36.354687 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:36.354844 (+ 157us) server_negotiation.cc:407] Connection header received
0809 12:22:36.355185 (+ 341us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:36.355191 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:36.355287 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:36.355411 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:36.356262 (+ 851us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38478: BlockingRecv error: recv got EOF from 127.0.0.1:38478 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 12:22:36.371824 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:36.364586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37584 (local address 127.19.183.253:43517)
0809 12:22:36.364799 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:36.364806 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:36.365394 (+ 588us) server_negotiation.cc:407] Connection header received
0809 12:22:36.365626 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:36.365634 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:36.365750 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:36.365856 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:22:36.366351 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:36.367561 (+ 1210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:36.369534 (+ 1973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:36.370332 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:36.370479 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:36.370985 (+ 506us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:22:36.371132 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:22:36.371249 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:36.371397 (+ 148us) server_negotiation.cc:299] Negotiation successful
0809 12:22:36.371484 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
W20260809 12:22:36.383208 21155 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:36.380769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38492 (local address 127.19.183.252:37575)
0809 12:22:36.381098 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:36.381104 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:36.381649 (+ 545us) server_negotiation.cc:407] Connection header received
0809 12:22:36.381860 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:36.381866 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:36.381974 (+ 108us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:36.382108 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:36.382975 (+ 867us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38492: BlockingRecv error: recv got EOF from 127.0.0.1:38492 (error 108)
Metrics: {"server-negotiator.queue_time_us":170}
I20260809 12:22:36.410286 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:36.404252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37590 (local address 127.19.183.253:43517)
0809 12:22:36.404486 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:36.404497 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:36.405061 (+ 564us) server_negotiation.cc:407] Connection header received
0809 12:22:36.405305 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:36.405309 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:36.405383 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:36.405487 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:22:36.405927 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:36.406760 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:36.408274 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:36.408812 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:36.408958 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:36.409425 (+ 467us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:22:36.409544 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:22:36.409660 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:36.409813 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 12:22:36.409893 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 12:22:36.419027 21155 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:36.416398 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38498 (local address 127.19.183.252:37575)
0809 12:22:36.416664 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:36.416673 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:36.417228 (+ 555us) server_negotiation.cc:407] Connection header received
0809 12:22:36.417536 (+ 308us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:36.417567 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:36.417666 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:36.417726 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:36.418748 (+ 1022us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38498: BlockingRecv error: recv got EOF from 127.0.0.1:38498 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 12:22:36.451653 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:36.444363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37606 (local address 127.19.183.253:43517)
0809 12:22:36.444577 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:36.444584 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:36.445157 (+ 573us) server_negotiation.cc:407] Connection header received
0809 12:22:36.445390 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:36.445396 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:36.445486 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:36.445595 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:22:36.446181 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:36.447379 (+ 1198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:36.449438 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:36.450080 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:36.450212 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:36.450853 (+ 641us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:22:36.450994 (+ 141us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:22:36.451090 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:36.451284 (+ 194us) server_negotiation.cc:299] Negotiation successful
0809 12:22:36.451373 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
W20260809 12:22:36.460791 21155 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:36.458313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38508 (local address 127.19.183.252:37575)
0809 12:22:36.458532 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:36.458538 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:36.459266 (+ 728us) server_negotiation.cc:407] Connection header received
0809 12:22:36.459446 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:36.459450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:36.459535 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:36.459617 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:36.460550 (+ 933us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38508: BlockingRecv error: recv got EOF from 127.0.0.1:38508 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 12:22:36.503077 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:36.497348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37614 (local address 127.19.183.253:43517)
0809 12:22:36.497640 (+ 292us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:36.497647 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:36.498088 (+ 441us) server_negotiation.cc:407] Connection header received
0809 12:22:36.498258 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:36.498264 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:36.498370 (+ 106us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:36.498494 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:22:36.499111 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:36.499884 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:36.501228 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:36.501702 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:36.501854 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:36.502276 (+ 422us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:22:36.502397 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:22:36.502506 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:36.502663 (+ 157us) server_negotiation.cc:299] Negotiation successful
0809 12:22:36.502774 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
W20260809 12:22:36.511884 21155 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:36.509754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38510 (local address 127.19.183.252:37575)
0809 12:22:36.509975 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:36.509981 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:36.510281 (+ 300us) server_negotiation.cc:407] Connection header received
0809 12:22:36.510460 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:36.510464 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:36.510549 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:36.510641 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:36.511607 (+ 966us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38510: BlockingRecv error: recv got EOF from 127.0.0.1:38510 (error 108)
Metrics: {"server-negotiator.queue_time_us":86}
I20260809 12:22:36.584682 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:36.579271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37630 (local address 127.19.183.253:43517)
0809 12:22:36.579537 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:36.579545 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:36.579900 (+ 355us) server_negotiation.cc:407] Connection header received
0809 12:22:36.580146 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:36.580150 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:36.580229 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:36.580333 (+ 104us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:22:36.580763 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:36.581490 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:36.582878 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:36.583321 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:36.583454 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:36.583947 (+ 493us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:22:36.584060 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:22:36.584180 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:36.584321 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 12:22:36.584404 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
W20260809 12:22:36.591913 21155 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:36.590172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38526 (local address 127.19.183.252:37575)
0809 12:22:36.590327 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:36.590331 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:36.590690 (+ 359us) server_negotiation.cc:407] Connection header received
0809 12:22:36.590920 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:36.590924 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:36.590996 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:36.591082 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:36.591749 (+ 667us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38526: BlockingRecv error: recv got EOF from 127.0.0.1:38526 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 12:22:36.728696 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:36.723309 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37640 (local address 127.19.183.253:43517)
0809 12:22:36.723521 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:36.723529 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:36.723884 (+ 355us) server_negotiation.cc:407] Connection header received
0809 12:22:36.724123 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:36.724128 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:36.724210 (+ 82us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:36.724316 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:22:36.724784 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:36.725467 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:36.726826 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:36.727249 (+ 423us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:36.727424 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:36.727927 (+ 503us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:22:36.728034 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:22:36.728152 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:36.728322 (+ 170us) server_negotiation.cc:299] Negotiation successful
0809 12:22:36.728402 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260809 12:22:36.736852 21155 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:36.734889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38538 (local address 127.19.183.252:37575)
0809 12:22:36.735114 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:36.735121 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:36.735359 (+ 238us) server_negotiation.cc:407] Connection header received
0809 12:22:36.735565 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:36.735572 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:36.735669 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:36.735781 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:36.736648 (+ 867us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38538: BlockingRecv error: recv got EOF from 127.0.0.1:38538 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260809 12:22:36.833081 21245 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b32952f2eeca447eab9ee78f68c7e5d9: acquiring CA information for follower catalog manager: success
I20260809 12:22:36.835035 21245 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b32952f2eeca447eab9ee78f68c7e5d9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 12:22:37.009593 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:37.004384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37646 (local address 127.19.183.253:43517)
0809 12:22:37.004645 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:37.004651 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:37.005168 (+ 517us) server_negotiation.cc:407] Connection header received
0809 12:22:37.005421 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:37.005429 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:37.005525 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:37.005610 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:22:37.006027 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:37.006823 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:37.008145 (+ 1322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:37.008567 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:37.008677 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:37.009020 (+ 343us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:22:37.009127 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:22:37.009196 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:37.009334 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 12:22:37.009398 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260809 12:22:37.021245 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:37.015657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38540 (local address 127.19.183.252:37575)
0809 12:22:37.015821 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:37.015825 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:37.016107 (+ 282us) server_negotiation.cc:407] Connection header received
0809 12:22:37.016237 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:37.016240 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:37.016402 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:37.016509 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:22:37.016886 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:37.017568 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:37.019057 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:37.019530 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:37.019633 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:37.020118 (+ 485us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:22:37.020736 (+ 618us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:22:37.020846 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:37.020940 (+ 94us) server_negotiation.cc:299] Negotiation successful
0809 12:22:37.021037 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 12:22:37.035876 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:37.028644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38546 (local address 127.19.183.252:37575)
0809 12:22:37.028855 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:37.028864 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:37.029779 (+ 915us) server_negotiation.cc:407] Connection header received
0809 12:22:37.029934 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:37.029937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:37.029994 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:37.030065 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:22:37.031178 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:37.032260 (+ 1082us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:37.034055 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:37.034569 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:37.034692 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:37.035151 (+ 459us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:22:37.035283 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:22:37.035373 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:37.035579 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 12:22:37.035669 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260809 12:22:37.036809 21042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:37.027565 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37650 (local address 127.19.183.253:43517)
0809 12:22:37.027864 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:37.027872 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:37.029406 (+ 1534us) server_negotiation.cc:407] Connection header received
0809 12:22:37.029710 (+ 304us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:37.029719 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:37.029810 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:37.029912 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:22:37.031055 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:37.031818 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:37.034812 (+ 2994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:37.035416 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:37.035567 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:37.036000 (+ 433us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:22:37.036123 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:22:37.036209 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:37.036406 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 12:22:37.036486 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":156}
I20260809 12:22:37.037892 21426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:37.027573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38640 (local address 127.19.183.254:33983)
0809 12:22:37.028985 (+ 1412us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:37.028993 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:37.029075 (+ 82us) server_negotiation.cc:407] Connection header received
0809 12:22:37.029212 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:37.029218 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:37.029451 (+ 233us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:37.029645 (+ 194us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:22:37.030969 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:37.032141 (+ 1172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:37.034385 (+ 2244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:37.035058 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:37.035173 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:37.036103 (+ 930us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:22:37.036532 (+ 429us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:22:37.036647 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:37.037606 (+ 959us) server_negotiation.cc:299] Negotiation successful
0809 12:22:37.037672 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1279,"thread_start_us":1188,"threads_started":1}
I20260809 12:22:37.043582 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 20834
I20260809 12:22:37.044548 20924 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:22:37.160039 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 20834
I20260809 12:22:37.192183 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 20940
I20260809 12:22:37.193360 21024 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:22:37.302536 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 20940
I20260809 12:22:37.333582 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 21046
I20260809 12:22:37.334775 21143 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:22:37.442013 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 21046
2026-08-09T12:22:37Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (5118 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 12:22:37 dist-test-slave-39l8 krb5kdc[21440](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 12:22:37 dist-test-slave-39l8 krb5kdc[21440](info): set up 2 sockets
Aug 09 12:22:37 dist-test-slave-39l8 krb5kdc[21440](info): commencing operation
krb5kdc: starting...
W20260809 12:22:38.755607 20191 mini_kdc.cc:121] Time spent starting KDC: real 1.265s user 0.002s sys 0.049s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/test-user.keytab.
Aug 09 12:22:38 dist-test-slave-39l8 krb5kdc[21440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T12:22:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T12:22:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.19.183.254@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.254@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
Entry for principal kudu/127.19.183.254 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab.
I20260809 12:22:39.108134 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:40117
--webserver_interface=127.19.183.254
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:37357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:40117,127.19.183.253:39393,127.19.183.252:36651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 12:22:39.455937 21456 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 12:22:39.456249 21456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:22:39.456324 21456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:22:39.456408 21456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:22:39.463467 21456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:22:39.463553 21456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:22:39.463611 21456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:22:39.463663 21456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:22:39.463721 21456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:22:39.473197 21456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:37357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:40117,127.19.183.253:39393,127.19.183.252:36651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.254.keytab
--principal=kudu/127.19.183.254
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.254:40117
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.19.183.254
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:22:39.474368 21456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:22:39.475560 21456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:22:39.487001 21468 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:39.487046 21463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:39.487089 21462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:22:39.488039 21456 server_base.cc:1034] running on GCE node
I20260809 12:22:39.488734 21456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:22:39.489992 21456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:22:39.491228 21456 hybrid_clock.cc:648] HybridClock initialized: now 1786278159491157 us; error 39 us; skew 500 ppm
I20260809 12:22:39.491582 21456 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:22:39 dist-test-slave-39l8 krb5kdc[21440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278159, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:22:39.494634 21456 init.cc:377] Logged in from keytab as kudu/127.19.183.254@KRBTEST.COM (short username kudu)
I20260809 12:22:39.496089 21456 webserver.cc:460] Webserver started at http://127.19.183.254:38567/ using document root <none> and password file <none>
I20260809 12:22:39.496809 21456 fs_manager.cc:359] Metadata directory not provided
I20260809 12:22:39.496940 21456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:22:39.497287 21456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:22:39.500319 21456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-0/data/instance:
uuid: "82171170d65740648e6eb14e2fe2c9c4"
format_stamp: "Formatted at 2026-08-09 12:22:39 on dist-test-slave-39l8"
server_key: "11654d02ab6138b813074356f2e7504b"
server_key_iv: "96927e8a4272f213c4b49b58950118a7"
server_key_version: "encryptionkey@0"
I20260809 12:22:39.501135 21456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-0/wal/instance:
uuid: "82171170d65740648e6eb14e2fe2c9c4"
format_stamp: "Formatted at 2026-08-09 12:22:39 on dist-test-slave-39l8"
server_key: "11654d02ab6138b813074356f2e7504b"
server_key_iv: "96927e8a4272f213c4b49b58950118a7"
server_key_version: "encryptionkey@0"
I20260809 12:22:39.506201 21456 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.002s
I20260809 12:22:39.509687 21477 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:22:39.511003 21456 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260809 12:22:39.511188 21456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-0/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-0/wal
uuid: "82171170d65740648e6eb14e2fe2c9c4"
format_stamp: "Formatted at 2026-08-09 12:22:39 on dist-test-slave-39l8"
server_key: "11654d02ab6138b813074356f2e7504b"
server_key_iv: "96927e8a4272f213c4b49b58950118a7"
server_key_version: "encryptionkey@0"
I20260809 12:22:39.511401 21456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:22:39.524466 21456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:22:39.528198 21456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:22:39.528458 21456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:22:39.547212 21456 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.254:40117
I20260809 12:22:39.547245 21547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.254:40117 every 8 connection(s)
I20260809 12:22:39.548489 21456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-0/data/info.pb
I20260809 12:22:39.551973 21548 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:22:39.554162 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 21456
I20260809 12:22:39.554349 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-0/wal/instance
I20260809 12:22:39.554955 20191 external_mini_cluster.cc:1428] Setting key 3b4f6728814b1292392d697cd8cd7a61
I20260809 12:22:39.561177 21548 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40117 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:22:39 dist-test-slave-39l8 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278159, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:22:39.586441 21551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:39.564984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57338 (local address 127.19.183.254:40117)
0809 12:22:39.566856 (+ 1872us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:39.566862 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:39.566902 (+ 40us) server_negotiation.cc:407] Connection header received
0809 12:22:39.569120 (+ 2218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:39.569125 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:39.569332 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:39.569496 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:39.571131 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:39.572317 (+ 1186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:39.574491 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:39.575105 (+ 614us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:39.575241 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:39.579436 (+ 4195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:39.579444 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:39.579456 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:39.579485 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:39.582175 (+ 2690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:39.582916 (+ 741us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:39.582923 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:39.582925 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:39.583062 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:39.583484 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:39.583494 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:39.583497 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:39.584007 (+ 510us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:39.584225 (+ 218us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:39.584751 (+ 526us) server_negotiation.cc:299] Negotiation successful
0809 12:22:39.585035 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1628,"spinlock_wait_cycles":15616,"thread_start_us":1405,"threads_started":1}
I20260809 12:22:39.586473 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:39.564984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:40117 (local address 127.0.0.1:57338)
0809 12:22:39.565889 (+ 905us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:39.565950 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0809 12:22:39.568756 (+ 2806us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:22:39.569578 (+ 822us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:22:39.569587 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:22:39.570383 (+ 796us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:22:39.570904 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:39.570931 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:39.572487 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:39.572492 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:39.574333 (+ 1841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:39.574351 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:39.575393 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:39.575408 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:39.575655 (+ 247us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:39.576351 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:22:39.576401 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:22:39.579206 (+ 2805us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:22:39.582318 (+ 3112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:39.582331 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:39.582342 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:39.582773 (+ 431us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:39.583190 (+ 417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:39.583195 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:39.583198 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:39.583366 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:39.584155 (+ 789us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:22:39.584163 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:22:39.584376 (+ 213us) client_negotiation.cc:769] Sending connection context
0809 12:22:39.584609 (+ 233us) client_negotiation.cc:240] Negotiation successful
0809 12:22:39.585186 (+ 577us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":605,"mutex_wait_us":143,"thread_start_us":376,"threads_started":1}
I20260809 12:22:39.589761 21548 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:22:39.590945 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:39.590531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:39393 (local address 127.0.0.1:37390)
0809 12:22:39.590700 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:39.590840 (+ 140us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:39393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 12:22:39.591225 21481 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.183.253:39393: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.19.183.253@KRBTEST.COM; defaulting to no policy
I20260809 12:22:39.592913 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:39.592552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:39393 (local address 127.0.0.1:37398)
0809 12:22:39.592720 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:39.592787 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:39393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260809 12:22:39.593122 21548 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.253:39393: Network error: Client connection negotiation failed: client connection to 127.19.183.253:39393: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.253@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.253 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.253.keytab.
Entry for principal kudu/127.19.183.253 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.253.keytab.
I20260809 12:22:39.629686 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.253:39393
--webserver_interface=127.19.183.253
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.253.keytab
--principal=kudu/127.19.183.253
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:37357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:40117,127.19.183.253:39393,127.19.183.252:36651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:22:39.642449 21548 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } attempt: 1
I20260809 12:22:39.643795 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:39.643309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:39393 (local address 127.0.0.1:37404)
0809 12:22:39.643517 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:39.643659 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:39393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 12:22:39.644824 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:39.644447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:39393 (local address 127.0.0.1:37410)
0809 12:22:39.644632 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:39.644698 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:39393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260809 12:22:39.645010 21548 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.253:39393: Network error: Client connection negotiation failed: client connection to 127.19.183.253:39393: connect: Connection refused (error 111)
I20260809 12:22:39.713254 21548 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } attempt: 2
I20260809 12:22:39.714517 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:39.713963 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:39393 (local address 127.0.0.1:37426)
0809 12:22:39.714228 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:39.714319 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:39393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":68}
I20260809 12:22:39.715587 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:39.715172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:39393 (local address 127.0.0.1:37430)
0809 12:22:39.715331 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:39.715419 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:39393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260809 12:22:39.715785 21548 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.253:39393: Network error: Client connection negotiation failed: client connection to 127.19.183.253:39393: connect: Connection refused (error 111)
I20260809 12:22:39.807163 21548 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } attempt: 3
I20260809 12:22:39.808380 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:39.807931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:39393 (local address 127.0.0.1:37442)
0809 12:22:39.808144 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:39.808249 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:39393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260809 12:22:39.809394 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:39.809043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:39393 (local address 127.0.0.1:37450)
0809 12:22:39.809190 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:39.809263 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:39393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260809 12:22:39.809584 21548 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.253:39393: Network error: Client connection negotiation failed: client connection to 127.19.183.253:39393: connect: Connection refused (error 111)
I20260809 12:22:39.952879 21548 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } attempt: 4
I20260809 12:22:39.954107 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:39.953557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:39393 (local address 127.0.0.1:37460)
0809 12:22:39.953881 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:39.953968 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:39393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163}
I20260809 12:22:39.955232 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:39.954874 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:39393 (local address 127.0.0.1:37472)
0809 12:22:39.955042 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:39.955119 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.253:39393: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260809 12:22:39.955502 21548 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.253:39393: Network error: Client connection negotiation failed: client connection to 127.19.183.253:39393: connect: Connection refused (error 111)
W20260809 12:22:39.977593 21556 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 12:22:39.977938 21556 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:22:39.978055 21556 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:22:39.978191 21556 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:22:39.985539 21556 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:22:39.985625 21556 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:22:39.985738 21556 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:22:39.985826 21556 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:22:39.985940 21556 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:22:39.995810 21556 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:37357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:40117,127.19.183.253:39393,127.19.183.252:36651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.253.keytab
--principal=kudu/127.19.183.253
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.253:39393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.19.183.253
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:22:39.997139 21556 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:22:39.998437 21556 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:22:40.010622 21563 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:40.010878 21562 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:40.010859 21574 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:22:40.011739 21556 server_base.cc:1034] running on GCE node
I20260809 12:22:40.012571 21556 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:22:40.014151 21556 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:22:40.017575 21556 hybrid_clock.cc:648] HybridClock initialized: now 1786278160017451 us; error 96 us; skew 500 ppm
I20260809 12:22:40.017953 21556 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:22:40 dist-test-slave-39l8 krb5kdc[21440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278160, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.253@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:22:40.022137 21556 init.cc:377] Logged in from keytab as kudu/127.19.183.253@KRBTEST.COM (short username kudu)
I20260809 12:22:40.023772 21556 webserver.cc:460] Webserver started at http://127.19.183.253:40113/ using document root <none> and password file <none>
I20260809 12:22:40.024475 21556 fs_manager.cc:359] Metadata directory not provided
I20260809 12:22:40.024616 21556 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:22:40.024982 21556 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:22:40.027981 21556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-1/data/instance:
uuid: "018b8ed1068643eeb0cd290105fc749a"
format_stamp: "Formatted at 2026-08-09 12:22:40 on dist-test-slave-39l8"
server_key: "82ca1ccf9fe9f1354bd08b08e0cbd032"
server_key_iv: "46197ef4f1c5fa98bfcf396390bee4ed"
server_key_version: "encryptionkey@0"
I20260809 12:22:40.028762 21556 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-1/wal/instance:
uuid: "018b8ed1068643eeb0cd290105fc749a"
format_stamp: "Formatted at 2026-08-09 12:22:40 on dist-test-slave-39l8"
server_key: "82ca1ccf9fe9f1354bd08b08e0cbd032"
server_key_iv: "46197ef4f1c5fa98bfcf396390bee4ed"
server_key_version: "encryptionkey@0"
I20260809 12:22:40.033725 21556 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.003s
I20260809 12:22:40.037060 21584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:22:40.038293 21556 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:22:40.038480 21556 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-1/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-1/wal
uuid: "018b8ed1068643eeb0cd290105fc749a"
format_stamp: "Formatted at 2026-08-09 12:22:40 on dist-test-slave-39l8"
server_key: "82ca1ccf9fe9f1354bd08b08e0cbd032"
server_key_iv: "46197ef4f1c5fa98bfcf396390bee4ed"
server_key_version: "encryptionkey@0"
I20260809 12:22:40.038698 21556 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:22:40.052645 21556 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:22:40.056998 21556 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:22:40.057282 21556 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:22:40.076558 21556 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.253:39393
I20260809 12:22:40.076577 21657 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.253:39393 every 8 connection(s)
I20260809 12:22:40.077909 21556 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-1/data/info.pb
I20260809 12:22:40.081471 21658 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:22:40.082239 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 21556
I20260809 12:22:40.082433 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-1/wal/instance
I20260809 12:22:40.082994 20191 external_mini_cluster.cc:1428] Setting key a8e036e5b5c3db1f61faa122cae1fa18
I20260809 12:22:40.088080 21658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40117 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:22:40 dist-test-slave-39l8 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278160, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.253@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:22:40.119022 21660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.090488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57344 (local address 127.19.183.254:40117)
0809 12:22:40.099826 (+ 9338us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:40.099831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:40.099851 (+ 20us) server_negotiation.cc:407] Connection header received
0809 12:22:40.102174 (+ 2323us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:40.102178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:40.102239 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:40.102334 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:40.104720 (+ 2386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.105444 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.108015 (+ 2571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.108426 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.108526 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:40.112851 (+ 4325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:40.112856 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:40.112859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:40.112907 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:40.115766 (+ 2859us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.116659 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.116665 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.116670 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.116795 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.117224 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.117229 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.117234 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.117543 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:40.117702 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:40.118391 (+ 689us) server_negotiation.cc:299] Negotiation successful
0809 12:22:40.118609 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9231,"thread_start_us":9137,"threads_started":1}
I20260809 12:22:40.119931 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.091237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:40117 (local address 127.0.0.1:57344)
0809 12:22:40.099039 (+ 7802us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.099093 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 12:22:40.101876 (+ 2783us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:22:40.102679 (+ 803us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:22:40.102694 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:22:40.103615 (+ 921us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:22:40.104399 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:40.104431 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.105619 (+ 1188us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.105627 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:40.107850 (+ 2223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:40.107871 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.108670 (+ 799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.108679 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:40.109036 (+ 357us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:40.109807 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:22:40.109848 (+ 41us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:22:40.112599 (+ 2751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:22:40.115924 (+ 3325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:40.115933 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:40.115948 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:40.116525 (+ 577us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:40.116909 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:40.116914 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:40.116918 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:40.117098 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:40.117682 (+ 584us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:22:40.117690 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:22:40.118024 (+ 334us) client_negotiation.cc:769] Sending connection context
0809 12:22:40.118284 (+ 260us) client_negotiation.cc:240] Negotiation successful
0809 12:22:40.118807 (+ 523us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7442,"thread_start_us":7188,"threads_started":1}
WARNING: no policy specified for kudu/127.19.183.252@KRBTEST.COM; defaulting to no policy
I20260809 12:22:40.121593 21658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.19.183.252@KRBTEST.COM" created.
Aug 09 12:22:40 dist-test-slave-39l8 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278160, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.253@KRBTEST.COM for kudu/127.19.183.253@KRBTEST.COM
I20260809 12:22:40.136328 21663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.122521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37484 (local address 127.19.183.253:39393)
0809 12:22:40.123234 (+ 713us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:40.123240 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:40.123265 (+ 25us) server_negotiation.cc:407] Connection header received
0809 12:22:40.123307 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:40.123312 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:40.123469 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:40.123690 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:40.124391 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.125434 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.126550 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.127129 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.127265 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:40.130687 (+ 3422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:40.130697 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:40.130760 (+ 63us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:40.130800 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:40.133422 (+ 2622us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.134236 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.134242 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.134244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.134353 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.134819 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.134822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.134824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.135133 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:40.135282 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:40.135865 (+ 583us) server_negotiation.cc:299] Negotiation successful
0809 12:22:40.136050 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":52,"server-negotiator.queue_time_us":548,"thread_start_us":376,"threads_started":1}
I20260809 12:22:40.136417 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.122283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:39393 (local address 127.0.0.1:37484)
0809 12:22:40.122480 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.122508 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 12:22:40.122639 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:22:40.123631 (+ 992us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:22:40.123636 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:22:40.124097 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:22:40.124216 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:40.124224 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.125578 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.125586 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:40.126409 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:40.126421 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.127273 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.127278 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:40.127502 (+ 224us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:40.128104 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:22:40.128120 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:22:40.130458 (+ 2338us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:22:40.133560 (+ 3102us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:40.133567 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:40.133574 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:40.134111 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:40.134497 (+ 386us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:40.134504 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:40.134510 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:40.134698 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:40.135273 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:22:40.135280 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:22:40.135440 (+ 160us) client_negotiation.cc:769] Sending connection context
0809 12:22:40.135596 (+ 156us) client_negotiation.cc:240] Negotiation successful
0809 12:22:40.135923 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":58,"mutex_wait_us":3}
I20260809 12:22:40.138779 21658 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:22:40.140025 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.139494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:36651 (local address 127.0.0.1:40502)
0809 12:22:40.139695 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.139838 (+ 143us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260809 12:22:40.140328 21589 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
I20260809 12:22:40.142073 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.141583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:36651 (local address 127.0.0.1:40514)
0809 12:22:40.141814 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.141938 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 12:22:40.142305 21658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:36651: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.19.183.252 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.252.keytab.
Entry for principal kudu/127.19.183.252 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.252.keytab.
I20260809 12:22:40.161988 20191 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
/tmp/dist-test-taskqUnAOm/build/asan/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.252:36651
--webserver_interface=127.19.183.252
--webserver_port=0
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.252.keytab
--principal=kudu/127.19.183.252
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.19.183.212:37357
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.19.183.254:40117,127.19.183.253:39393,127.19.183.252:36651
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 12:22:40.191661 21658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } attempt: 1
I20260809 12:22:40.193069 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.192436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:36651 (local address 127.0.0.1:40524)
0809 12:22:40.192710 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.192854 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 12:22:40.194157 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.193700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:36651 (local address 127.0.0.1:40540)
0809 12:22:40.193909 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.194006 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260809 12:22:40.194388 21658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:36651: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
I20260809 12:22:40.254783 21548 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } attempt: 5
I20260809 12:22:40.262636 21658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } attempt: 2
Aug 09 12:22:40 dist-test-slave-39l8 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278159, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for kudu/127.19.183.253@KRBTEST.COM
I20260809 12:22:40.263746 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.263222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:36651 (local address 127.0.0.1:40550)
0809 12:22:40.263450 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.263552 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":84}
I20260809 12:22:40.264945 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.264519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:36651 (local address 127.0.0.1:40552)
0809 12:22:40.264692 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.264785 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260809 12:22:40.265329 21658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:36651: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
I20260809 12:22:40.269917 21663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.255761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37498 (local address 127.19.183.253:39393)
0809 12:22:40.255986 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:40.255994 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:40.256022 (+ 28us) server_negotiation.cc:407] Connection header received
0809 12:22:40.256118 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:40.256125 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:40.256223 (+ 98us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:40.256420 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:40.257687 (+ 1267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.258852 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.259998 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.260672 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.260801 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:40.264229 (+ 3428us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:40.264234 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:40.264239 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:40.264279 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:40.267274 (+ 2995us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.268132 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.268136 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.268138 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.268252 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.268714 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.268719 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.268722 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.269014 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:40.269129 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:40.269573 (+ 444us) server_negotiation.cc:299] Negotiation successful
0809 12:22:40.269737 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260809 12:22:40.270117 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.255570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:39393 (local address 127.0.0.1:37498)
0809 12:22:40.255815 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.255838 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0809 12:22:40.255964 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:22:40.256500 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:22:40.256508 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:22:40.257254 (+ 746us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:22:40.257487 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:40.257497 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.258997 (+ 1500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.259001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:40.259846 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:40.259859 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.260819 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.260823 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:40.260993 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:40.261637 (+ 644us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:22:40.261655 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:22:40.263985 (+ 2330us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:22:40.267419 (+ 3434us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:40.267424 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:40.267428 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:40.268008 (+ 580us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:40.268376 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:40.268383 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:40.268389 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:40.268580 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:40.269140 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:22:40.269146 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:22:40.269314 (+ 168us) client_negotiation.cc:769] Sending connection context
0809 12:22:40.269468 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 12:22:40.269795 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":91}
I20260809 12:22:40.271193 21548 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:22:40.272316 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.271860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:36651 (local address 127.0.0.1:40560)
0809 12:22:40.272058 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.272151 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
I20260809 12:22:40.273386 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.272925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:36651 (local address 127.0.0.1:40572)
0809 12:22:40.273106 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.273228 (+ 122us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 12:22:40.273664 21548 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:36651: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
I20260809 12:22:40.324908 21548 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } attempt: 1
I20260809 12:22:40.326222 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.325590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:36651 (local address 127.0.0.1:40574)
0809 12:22:40.325874 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.326011 (+ 137us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260809 12:22:40.327386 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.327003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:36651 (local address 127.0.0.1:40576)
0809 12:22:40.327161 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.327245 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 12:22:40.327615 21548 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:36651: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
I20260809 12:22:40.356678 21658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } attempt: 3
I20260809 12:22:40.357857 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.357406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:36651 (local address 127.0.0.1:40584)
0809 12:22:40.357594 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.357703 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260809 12:22:40.359002 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.358539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:36651 (local address 127.0.0.1:40588)
0809 12:22:40.358820 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.358895 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190}
W20260809 12:22:40.359404 21658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:36651: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
I20260809 12:22:40.395865 21548 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } attempt: 2
I20260809 12:22:40.397187 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.396600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:36651 (local address 127.0.0.1:40592)
0809 12:22:40.396865 (+ 265us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.396977 (+ 112us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260809 12:22:40.398391 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.397933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:36651 (local address 127.0.0.1:40596)
0809 12:22:40.398131 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.398233 (+ 102us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260809 12:22:40.398670 21548 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:36651: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
I20260809 12:22:40.502612 21658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } attempt: 4
I20260809 12:22:40.503840 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.503360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:36651 (local address 127.0.0.1:40598)
0809 12:22:40.503584 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.503685 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260809 12:22:40.504843 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.504557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:36651 (local address 127.0.0.1:40614)
0809 12:22:40.504688 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.504750 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 12:22:40.504941 21548 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } attempt: 3
W20260809 12:22:40.505336 21658 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:36651: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
I20260809 12:22:40.506068 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.505596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:36651 (local address 127.0.0.1:40618)
0809 12:22:40.505793 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.505892 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
I20260809 12:22:40.507079 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.506627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:36651 (local address 127.0.0.1:40634)
0809 12:22:40.506856 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.506949 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106}
W20260809 12:22:40.507289 21548 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.19.183.252:36651: Network error: Client connection negotiation failed: client connection to 127.19.183.252:36651: connect: Connection refused (error 111)
W20260809 12:22:40.515268 21669 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 12:22:40.515540 21669 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 12:22:40.515614 21669 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 12:22:40.515697 21669 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 12:22:40.522840 21669 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 12:22:40.522913 21669 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 12:22:40.523034 21669 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 12:22:40.523125 21669 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 12:22:40.523219 21669 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 12:22:40.532991 21669 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.19.183.212:37357
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.19.183.254:40117,127.19.183.253:39393,127.19.183.252:36651
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/krb5kdc/kudu_127.19.183.252.keytab
--principal=kudu/127.19.183.252
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.19.183.252:36651
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.19.183.252
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type FASTDEBUG
built by systest at 09 Aug 2026 11:06:41 UTC on ccycloud.quasar-bwoogy.root.comops.site
build id 4575991
ASAN enabled
I20260809 12:22:40.534226 21669 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 12:22:40.535581 21669 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 12:22:40.547181 21690 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 12:22:40.547206 21682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:22:40.547951 21669 server_base.cc:1034] running on GCE node
W20260809 12:22:40.551194 21681 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 12:22:40.552075 21669 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 12:22:40.553413 21669 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 12:22:40.554665 21669 hybrid_clock.cc:648] HybridClock initialized: now 1786278160554592 us; error 44 us; skew 500 ppm
I20260809 12:22:40.555013 21669 server_base.cc:834] Flag tcmalloc_max_total_thread_cache_bytes is not working since tcmalloc is not enabled.
Aug 09 12:22:40 dist-test-slave-39l8 krb5kdc[21440](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278160, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.252@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 12:22:40.558043 21669 init.cc:377] Logged in from keytab as kudu/127.19.183.252@KRBTEST.COM (short username kudu)
I20260809 12:22:40.559412 21669 webserver.cc:460] Webserver started at http://127.19.183.252:42315/ using document root <none> and password file <none>
I20260809 12:22:40.560108 21669 fs_manager.cc:359] Metadata directory not provided
I20260809 12:22:40.560448 21669 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 12:22:40.560738 21669 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 12:22:40.563735 21669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-2/data/instance:
uuid: "68c457b0bd4e440194ea6ef9bcac91ab"
format_stamp: "Formatted at 2026-08-09 12:22:40 on dist-test-slave-39l8"
server_key: "511852da1a230362ac5148eedc13270d"
server_key_iv: "edefe4f14d02c8481ad7d6cdbf78630d"
server_key_version: "encryptionkey@0"
I20260809 12:22:40.564500 21669 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-2/wal/instance:
uuid: "68c457b0bd4e440194ea6ef9bcac91ab"
format_stamp: "Formatted at 2026-08-09 12:22:40 on dist-test-slave-39l8"
server_key: "511852da1a230362ac5148eedc13270d"
server_key_iv: "edefe4f14d02c8481ad7d6cdbf78630d"
server_key_version: "encryptionkey@0"
I20260809 12:22:40.569432 21669 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260809 12:22:40.572767 21697 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 12:22:40.574056 21669 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260809 12:22:40.574254 21669 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-2/data,/tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-2/wal
uuid: "68c457b0bd4e440194ea6ef9bcac91ab"
format_stamp: "Formatted at 2026-08-09 12:22:40 on dist-test-slave-39l8"
server_key: "511852da1a230362ac5148eedc13270d"
server_key_iv: "edefe4f14d02c8481ad7d6cdbf78630d"
server_key_version: "encryptionkey@0"
I20260809 12:22:40.574460 21669 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 12:22:40.592304 21669 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 12:22:40.596544 21669 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 12:22:40.596823 21669 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 12:22:40.615352 21669 rpc_server.cc:307] RPC server started. Bound to: 127.19.183.252:36651
I20260809 12:22:40.615371 21763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.19.183.252:36651 every 8 connection(s)
I20260809 12:22:40.616762 21669 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-2/data/info.pb
I20260809 12:22:40.620285 21764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 12:22:40.624598 20191 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu as pid 21669
I20260809 12:22:40.624795 20191 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskqUnAOm/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786277774779245-20191-0/minicluster-data/master-2/wal/instance
I20260809 12:22:40.625320 20191 external_mini_cluster.cc:1428] Setting key 7b3278f030092948867b62c4f6390d27
I20260809 12:22:40.628159 21764 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40117 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:22:40 dist-test-slave-39l8 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.252@KRBTEST.COM
Aug 09 12:22:40 dist-test-slave-39l8 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278160, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.252@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:22:40.649784 21771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.632116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57346 (local address 127.19.183.254:40117)
0809 12:22:40.632691 (+ 575us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:40.632701 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:40.633387 (+ 686us) server_negotiation.cc:407] Connection header received
0809 12:22:40.633587 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:40.633590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:40.633675 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:40.633763 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:40.635244 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.636332 (+ 1088us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.638328 (+ 1996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.638933 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.639063 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:40.642975 (+ 3912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:40.642982 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:40.642989 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:40.643065 (+ 76us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:40.646433 (+ 3368us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.647688 (+ 1255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.647692 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.647694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.647816 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.648232 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.648235 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.648237 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.648567 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:40.648742 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:40.649360 (+ 618us) server_negotiation.cc:299] Negotiation successful
0809 12:22:40.649513 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":399,"thread_start_us":267,"threads_started":1}
I20260809 12:22:40.650828 21768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.627139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40638 (local address 127.19.183.252:36651)
0809 12:22:40.627916 (+ 777us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:40.627928 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:40.627970 (+ 42us) server_negotiation.cc:407] Connection header received
0809 12:22:40.630053 (+ 2083us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:40.630075 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:40.630497 (+ 422us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:40.630874 (+ 377us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:40.632800 (+ 1926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.634977 (+ 2177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.636787 (+ 1810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.637849 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.638087 (+ 238us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:40.642399 (+ 4312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:40.642407 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:40.642417 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:40.642450 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:40.646170 (+ 3720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.647097 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.647105 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.647108 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.647322 (+ 214us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.647668 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.647672 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.647674 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.648278 (+ 604us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:40.648546 (+ 268us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:40.649017 (+ 471us) server_negotiation.cc:299] Negotiation successful
0809 12:22:40.649427 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":63,"server-negotiator.queue_time_us":583,"thread_start_us":366,"threads_started":1}
I20260809 12:22:40.650848 21772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.632094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.254:40117 (local address 127.0.0.1:57346)
0809 12:22:40.633212 (+ 1118us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.633274 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0809 12:22:40.633456 (+ 182us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:22:40.633867 (+ 411us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:22:40.633874 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:22:40.634698 (+ 824us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:22:40.635094 (+ 396us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:40.635110 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.636501 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.636512 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:40.638167 (+ 1655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:40.638183 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.639172 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.639177 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:40.639297 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:40.639923 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:22:40.639944 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:22:40.642688 (+ 2744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:22:40.646607 (+ 3919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:40.646618 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:40.646666 (+ 48us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:40.647322 (+ 656us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:40.647929 (+ 607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:40.647934 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:40.647937 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:40.648117 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:40.648745 (+ 628us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:22:40.648753 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:22:40.649023 (+ 270us) client_negotiation.cc:769] Sending connection context
0809 12:22:40.649217 (+ 194us) client_negotiation.cc:240] Negotiation successful
0809 12:22:40.649546 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"mutex_wait_us":122,"spinlock_wait_cycles":17280,"thread_start_us":682,"threads_started":1}
I20260809 12:22:40.652415 21764 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 12:22:40 dist-test-slave-39l8 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278160, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.252@KRBTEST.COM for kudu/127.19.183.253@KRBTEST.COM
Aug 09 12:22:40 dist-test-slave-39l8 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.254@KRBTEST.COM
I20260809 12:22:40.667080 21663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.653198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37500 (local address 127.19.183.253:39393)
0809 12:22:40.653369 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:40.653375 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:40.653508 (+ 133us) server_negotiation.cc:407] Connection header received
0809 12:22:40.653705 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:40.653710 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:40.653793 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:40.653897 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:40.654829 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.655593 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.657081 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.657482 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.657575 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:40.660828 (+ 3253us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:40.660837 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:40.660843 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:40.660885 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:40.664310 (+ 3425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.665104 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.665111 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.665115 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.665276 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.665669 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.665674 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.665677 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.666035 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:40.666197 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:40.666553 (+ 356us) server_negotiation.cc:299] Negotiation successful
0809 12:22:40.666763 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 12:22:40.667112 21772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.653271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.253:39393 (local address 127.0.0.1:37500)
0809 12:22:40.653409 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.653428 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:22:40.653564 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:22:40.654085 (+ 521us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:22:40.654090 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:22:40.654518 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:22:40.654656 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:40.654664 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.655747 (+ 1083us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.655751 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:40.656935 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:40.656951 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.657635 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.657642 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:40.657790 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:40.658303 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:22:40.658319 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:22:40.660616 (+ 2297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:22:40.664442 (+ 3826us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:40.664446 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:40.664448 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:40.664979 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:40.665400 (+ 421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:40.665403 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:40.665405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:40.665553 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:40.666151 (+ 598us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:22:40.666155 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:22:40.666280 (+ 125us) client_negotiation.cc:769] Sending connection context
0809 12:22:40.666457 (+ 177us) client_negotiation.cc:240] Negotiation successful
0809 12:22:40.666774 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 12:22:40.668313 21764 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } has no permanent_uuid. Determining permanent_uuid...
I20260809 12:22:40.669145 21771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.653751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57356 (local address 127.19.183.254:40117)
0809 12:22:40.653913 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:40.653918 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:40.653935 (+ 17us) server_negotiation.cc:407] Connection header received
0809 12:22:40.654085 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:40.654090 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:40.654175 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:40.654286 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:40.655426 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.656191 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.657689 (+ 1498us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.658276 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.658368 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:40.662896 (+ 4528us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:40.662904 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:40.662912 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:40.662956 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:40.666258 (+ 3302us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.667066 (+ 808us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.667074 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.667078 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.667227 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.667716 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.667723 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.667725 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.668101 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:40.668354 (+ 253us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:40.668733 (+ 379us) server_negotiation.cc:299] Negotiation successful
0809 12:22:40.668900 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 12:22:40 dist-test-slave-39l8 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278160, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.252@KRBTEST.COM for kudu/127.19.183.252@KRBTEST.COM
Aug 09 12:22:40 dist-test-slave-39l8 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278158, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.19.183.253@KRBTEST.COM
I20260809 12:22:40.681896 21772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.669035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:36651 (local address 127.0.0.1:40642)
0809 12:22:40.669658 (+ 623us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.669677 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:22:40.669791 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:22:40.670143 (+ 352us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:22:40.670149 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:22:40.670588 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:22:40.670797 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:40.670811 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.671784 (+ 973us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.671790 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:40.672629 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:40.672639 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.673302 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.673308 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:40.673556 (+ 248us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:40.674083 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:22:40.674098 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:22:40.675984 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:22:40.679440 (+ 3456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:40.679443 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:40.679446 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:40.679818 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:40.680244 (+ 426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:40.680249 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:40.680253 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:40.680399 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:40.680895 (+ 496us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:22:40.680900 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:22:40.681041 (+ 141us) client_negotiation.cc:769] Sending connection context
0809 12:22:40.681180 (+ 139us) client_negotiation.cc:240] Negotiation successful
0809 12:22:40.681474 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":522,"mutex_wait_us":42}
I20260809 12:22:40.681946 21768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.669395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40642 (local address 127.19.183.252:36651)
0809 12:22:40.669568 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:40.669574 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:40.669741 (+ 167us) server_negotiation.cc:407] Connection header received
0809 12:22:40.669911 (+ 170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:40.669917 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:40.669988 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:40.670078 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:40.670983 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.671658 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.672768 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.673162 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.673293 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:40.676165 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:40.676169 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:40.676172 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:40.676198 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:40.679299 (+ 3101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.679946 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.679952 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.679957 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.680114 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.680518 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.680523 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.680526 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.680775 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:40.680926 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:40.681264 (+ 338us) server_negotiation.cc:299] Negotiation successful
0809 12:22:40.681631 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":150,"server-negotiator.queue_time_us":53}
I20260809 12:22:40.684535 21548 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } attempt: 4
I20260809 12:22:40.686280 21663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.670814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37502 (local address 127.19.183.253:39393)
0809 12:22:40.670971 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:40.670976 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:40.670994 (+ 18us) server_negotiation.cc:407] Connection header received
0809 12:22:40.671153 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:40.671158 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:40.671222 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:40.671318 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:40.672703 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.673385 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.674732 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.675169 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.675306 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:40.680086 (+ 4780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:40.680091 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:40.680094 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:40.680130 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:40.683110 (+ 2980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.683870 (+ 760us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.683874 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.683876 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.683995 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.684417 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.684420 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.684422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.684658 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:40.684966 (+ 308us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:40.685717 (+ 751us) server_negotiation.cc:299] Negotiation successful
0809 12:22:40.685942 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260809 12:22:40.688493 21764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c457b0bd4e440194ea6ef9bcac91ab: Bootstrap starting.
I20260809 12:22:40.691226 20191 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
Aug 09 12:22:40 dist-test-slave-39l8 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278159, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.254@KRBTEST.COM for kudu/127.19.183.252@KRBTEST.COM
I20260809 12:22:40.693423 21764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 68c457b0bd4e440194ea6ef9bcac91ab: Neither blocks nor log segments found. Creating new log.
I20260809 12:22:40.694906 21764 log.cc:824] T 00000000000000000000000000000000 P 68c457b0bd4e440194ea6ef9bcac91ab: Log is configured to *not* fsync() on all Append() calls
I20260809 12:22:40.694394 21516 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "82171170d65740648e6eb14e2fe2c9c4"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57356
I20260809 12:22:40.698913 21764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 68c457b0bd4e440194ea6ef9bcac91ab: No bootstrap required, opened a new log
I20260809 12:22:40.701063 21550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.685120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:36651 (local address 127.0.0.1:40650)
0809 12:22:40.685314 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.685334 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 12:22:40.685436 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:22:40.686686 (+ 1250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:22:40.686691 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:22:40.687215 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:22:40.687427 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:40.687439 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.688772 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.688777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:40.690227 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:40.690241 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.691267 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.691273 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:40.691488 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:40.692060 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:22:40.692078 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:22:40.694165 (+ 2087us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:22:40.698149 (+ 3984us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:40.698155 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:40.698158 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:40.698668 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:40.699206 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:40.699211 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:40.699216 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:40.699393 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:40.700087 (+ 694us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:22:40.700092 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:22:40.700259 (+ 167us) client_negotiation.cc:769] Sending connection context
0809 12:22:40.700413 (+ 154us) client_negotiation.cc:240] Negotiation successful
0809 12:22:40.700681 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":65}
I20260809 12:22:40.701251 21768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.685296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40650 (local address 127.19.183.252:36651)
0809 12:22:40.686235 (+ 939us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:40.686241 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:40.686262 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:22:40.686313 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:40.686319 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:40.686522 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:40.686639 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:40.687584 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.688543 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.690374 (+ 1831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.691124 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.691256 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:40.694357 (+ 3101us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:40.694367 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:40.694373 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:40.694408 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:40.697934 (+ 3526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.698860 (+ 926us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.698864 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.698869 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.699079 (+ 210us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.699527 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.699535 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.699539 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.699939 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:40.700100 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:40.700598 (+ 498us) server_negotiation.cc:299] Negotiation successful
0809 12:22:40.700829 (+ 231us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":834}
I20260809 12:22:40.706967 21548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4: Bootstrap starting.
I20260809 12:22:40.709848 21764 raft_consensus.cc:348] T 00000000000000000000000000000000 P 68c457b0bd4e440194ea6ef9bcac91ab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82171170d65740648e6eb14e2fe2c9c4" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40117 } } peers { permanent_uuid: "018b8ed1068643eeb0cd290105fc749a" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } } peers { permanent_uuid: "68c457b0bd4e440194ea6ef9bcac91ab" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } }
I20260809 12:22:40.710327 21764 raft_consensus.cc:729] T 00000000000000000000000000000000 P 68c457b0bd4e440194ea6ef9bcac91ab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 68c457b0bd4e440194ea6ef9bcac91ab, State: Initialized, Role: FOLLOWER
I20260809 12:22:40.710950 21548 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4: Neither blocks nor log segments found. Creating new log.
I20260809 12:22:40.710901 21764 consensus_queue.cc:260] T 00000000000000000000000000000000 P 68c457b0bd4e440194ea6ef9bcac91ab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82171170d65740648e6eb14e2fe2c9c4" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40117 } } peers { permanent_uuid: "018b8ed1068643eeb0cd290105fc749a" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } } peers { permanent_uuid: "68c457b0bd4e440194ea6ef9bcac91ab" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } }
I20260809 12:22:40.711948 21776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c457b0bd4e440194ea6ef9bcac91ab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82171170d65740648e6eb14e2fe2c9c4" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40117 } } peers { permanent_uuid: "018b8ed1068643eeb0cd290105fc749a" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } } peers { permanent_uuid: "68c457b0bd4e440194ea6ef9bcac91ab" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } } }
I20260809 12:22:40.712384 21776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c457b0bd4e440194ea6ef9bcac91ab [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:22:40.712394 21548 log.cc:824] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4: Log is configured to *not* fsync() on all Append() calls
I20260809 12:22:40.713269 21764 sys_catalog.cc:564] T 00000000000000000000000000000000 P 68c457b0bd4e440194ea6ef9bcac91ab [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:22:40.715842 21548 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4: No bootstrap required, opened a new log
W20260809 12:22:40.723786 21791 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 68c457b0bd4e440194ea6ef9bcac91ab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 12:22:40.723917 21791 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 12:22:40.723739 21548 raft_consensus.cc:348] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82171170d65740648e6eb14e2fe2c9c4" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40117 } } peers { permanent_uuid: "018b8ed1068643eeb0cd290105fc749a" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } } peers { permanent_uuid: "68c457b0bd4e440194ea6ef9bcac91ab" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } }
I20260809 12:22:40.724140 21548 raft_consensus.cc:729] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82171170d65740648e6eb14e2fe2c9c4, State: Initialized, Role: FOLLOWER
I20260809 12:22:40.724625 21548 consensus_queue.cc:260] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82171170d65740648e6eb14e2fe2c9c4" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40117 } } peers { permanent_uuid: "018b8ed1068643eeb0cd290105fc749a" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } } peers { permanent_uuid: "68c457b0bd4e440194ea6ef9bcac91ab" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } }
I20260809 12:22:40.727492 21793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82171170d65740648e6eb14e2fe2c9c4" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40117 } } peers { permanent_uuid: "018b8ed1068643eeb0cd290105fc749a" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } } peers { permanent_uuid: "68c457b0bd4e440194ea6ef9bcac91ab" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } } }
I20260809 12:22:40.728279 21793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:22:40.728646 21548 sys_catalog.cc:564] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:22:40.728729 21516 raft_consensus.cc:482] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 12:22:40.728917 21516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:22:40.732515 21516 raft_consensus.cc:504] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82171170d65740648e6eb14e2fe2c9c4" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40117 } } peers { permanent_uuid: "018b8ed1068643eeb0cd290105fc749a" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } } peers { permanent_uuid: "68c457b0bd4e440194ea6ef9bcac91ab" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } }
I20260809 12:22:40.734274 21516 leader_election.cc:288] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4 [CANDIDATE]: Term 1 election: Requested vote from peers 018b8ed1068643eeb0cd290105fc749a (127.19.183.253:39393), 68c457b0bd4e440194ea6ef9bcac91ab (127.19.183.252:36651)
I20260809 12:22:40.721432 21669 master_runner.cc:186] Error getting master registration for 127.19.183.254:40117: OK, instance_id { permanent_uuid: "82171170d65740648e6eb14e2fe2c9c4" instance_seqno: 1786278159541927 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 12:22:40.734766 21669 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 12:22:40.734964 21732 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "82171170d65740648e6eb14e2fe2c9c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "68c457b0bd4e440194ea6ef9bcac91ab"
I20260809 12:22:40.735460 21732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 68c457b0bd4e440194ea6ef9bcac91ab [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:22:40.739044 21732 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 68c457b0bd4e440194ea6ef9bcac91ab [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82171170d65740648e6eb14e2fe2c9c4 in term 1.
I20260809 12:22:40.739667 21482 leader_election.cc:302] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 68c457b0bd4e440194ea6ef9bcac91ab, 82171170d65740648e6eb14e2fe2c9c4; no voters:
I20260809 12:22:40.740176 21793 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 12:22:40.751013 21793 raft_consensus.cc:686] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4 [term 1 LEADER]: Becoming Leader. State: Replica: 82171170d65740648e6eb14e2fe2c9c4, State: Running, Role: LEADER
I20260809 12:22:40.751752 21793 consensus_queue.cc:237] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82171170d65740648e6eb14e2fe2c9c4" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40117 } } peers { permanent_uuid: "018b8ed1068643eeb0cd290105fc749a" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } } peers { permanent_uuid: "68c457b0bd4e440194ea6ef9bcac91ab" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } }
I20260809 12:22:40.761353 21797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82171170d65740648e6eb14e2fe2c9c4. Latest consensus state: current_term: 1 leader_uuid: "82171170d65740648e6eb14e2fe2c9c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82171170d65740648e6eb14e2fe2c9c4" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40117 } } peers { permanent_uuid: "018b8ed1068643eeb0cd290105fc749a" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } } peers { permanent_uuid: "68c457b0bd4e440194ea6ef9bcac91ab" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } } }
I20260809 12:22:40.761847 21797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:40.769601 21811 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
W20260809 12:22:40.781111 21485 tablet.cc:2369] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:22:40.781653 21811 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 12:22:40.804520 21658 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } attempt: 5
I20260809 12:22:40.810657 21732 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 68c457b0bd4e440194ea6ef9bcac91ab [term 1 FOLLOWER]: Refusing update from remote peer 82171170d65740648e6eb14e2fe2c9c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:22:40.811805 21797 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "68c457b0bd4e440194ea6ef9bcac91ab" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
Aug 09 12:22:40 dist-test-slave-39l8 krb5kdc[21440](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786278160, etypes {rep=17 tkt=17 ses=17}, kudu/127.19.183.253@KRBTEST.COM for kudu/127.19.183.252@KRBTEST.COM
I20260809 12:22:40.834828 21771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.753335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57370 (local address 127.19.183.254:40117)
0809 12:22:40.753561 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:40.753569 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:40.771281 (+ 17712us) server_negotiation.cc:407] Connection header received
0809 12:22:40.771610 (+ 329us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:40.771618 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:40.771727 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:40.771814 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:40.774593 (+ 2779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.775987 (+ 1394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.782015 (+ 6028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.782522 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.782606 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:40.823634 (+ 41028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:40.823643 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:40.823649 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:40.823715 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:40.828108 (+ 4393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.832089 (+ 3981us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.832097 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.832101 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.832259 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.832774 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.832779 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.832781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.833250 (+ 469us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:40.833555 (+ 305us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:40.834130 (+ 575us) server_negotiation.cc:299] Negotiation successful
0809 12:22:40.834360 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 12:22:40.849012 21704 tablet.cc:2369] T 00000000000000000000000000000000 P 68c457b0bd4e440194ea6ef9bcac91ab: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 12:22:40.849891 21663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.759064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37508 (local address 127.19.183.253:39393)
0809 12:22:40.759250 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:40.759258 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:40.776173 (+ 16915us) server_negotiation.cc:407] Connection header received
0809 12:22:40.776262 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:40.776271 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:40.776381 (+ 110us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:40.776499 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:40.783715 (+ 7216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.784806 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.791245 (+ 6439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.791915 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.792043 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:40.831175 (+ 39132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:40.831183 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:40.831188 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:40.831225 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:40.834767 (+ 3542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.843033 (+ 8266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.843038 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.843042 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.843186 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.845104 (+ 1918us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.845109 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.845113 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.845493 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:40.845773 (+ 280us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:40.849251 (+ 3478us) server_negotiation.cc:299] Negotiation successful
0809 12:22:40.849500 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 12:22:40.855190 21661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.805160 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.19.183.252:36651 (local address 127.0.0.1:40674)
0809 12:22:40.805551 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0809 12:22:40.805570 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0809 12:22:40.805690 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 12:22:40.810867 (+ 5177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 12:22:40.810875 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 12:22:40.811569 (+ 694us) client_negotiation.cc:189] Negotiated authn=SASL
0809 12:22:40.811842 (+ 273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:40.811859 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.817636 (+ 5777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.817645 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:40.819606 (+ 1961us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 12:22:40.819628 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.824737 (+ 5109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.824747 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 12:22:40.825049 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:40.825749 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 12:22:40.825781 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0809 12:22:40.830048 (+ 4267us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 12:22:40.852214 (+ 22166us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:40.852219 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:40.852223 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:40.852817 (+ 594us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:40.853341 (+ 524us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 12:22:40.853346 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 12:22:40.853350 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 12:22:40.853525 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 12:22:40.854209 (+ 684us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 12:22:40.854213 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 12:22:40.854361 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 12:22:40.854516 (+ 155us) client_negotiation.cc:240] Negotiation successful
0809 12:22:40.854837 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263}
I20260809 12:22:40.855644 21776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c457b0bd4e440194ea6ef9bcac91ab [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82171170d65740648e6eb14e2fe2c9c4. Latest consensus state: current_term: 1 leader_uuid: "82171170d65740648e6eb14e2fe2c9c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82171170d65740648e6eb14e2fe2c9c4" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40117 } } peers { permanent_uuid: "018b8ed1068643eeb0cd290105fc749a" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } } peers { permanent_uuid: "68c457b0bd4e440194ea6ef9bcac91ab" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } } }
I20260809 12:22:40.856475 21776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c457b0bd4e440194ea6ef9bcac91ab [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:22:40.856642 21819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.805921 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40674 (local address 127.19.183.252:36651)
0809 12:22:40.810049 (+ 4128us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:40.810059 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:40.810129 (+ 70us) server_negotiation.cc:407] Connection header received
0809 12:22:40.810275 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:40.810284 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:40.810485 (+ 201us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:40.812121 (+ 1636us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:40.812792 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.817173 (+ 4381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.820204 (+ 3031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.824293 (+ 4089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.830557 (+ 6264us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:40.831825 (+ 1268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:40.831836 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:40.832034 (+ 198us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:40.832258 (+ 224us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:40.852033 (+ 19775us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.853004 (+ 971us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.853013 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.853017 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.853193 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.853678 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.853683 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.853686 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.854036 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:40.855433 (+ 1397us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:40.855579 (+ 146us) server_negotiation.cc:299] Negotiation successful
0809 12:22:40.856178 (+ 599us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3790,"thread_start_us":3479,"threads_started":1}
I20260809 12:22:40.863124 21658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 018b8ed1068643eeb0cd290105fc749a: Bootstrap starting.
I20260809 12:22:40.863533 21768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:40.770631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40662 (local address 127.19.183.252:36651)
0809 12:22:40.770845 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:40.770852 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:40.781909 (+ 11057us) server_negotiation.cc:407] Connection header received
0809 12:22:40.785015 (+ 3106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:40.785021 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:40.785144 (+ 123us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:40.785243 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:40.794394 (+ 9151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.795487 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.820648 (+ 25161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:40.821450 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:40.821552 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:40.855199 (+ 33647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 12:22:40.855208 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 12:22:40.855214 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 12:22:40.855261 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0809 12:22:40.859033 (+ 3772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.860384 (+ 1351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.860390 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.860394 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.860565 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 12:22:40.861201 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 12:22:40.861206 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 12:22:40.861208 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 12:22:40.861600 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 12:22:40.861857 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:40.862764 (+ 907us) server_negotiation.cc:299] Negotiation successful
0809 12:22:40.862994 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260809 12:22:40.873878 21816 mvcc.cc:205] Tried to move back new op lower bound from 7316595346627928064 to 7316595346446372864. Current Snapshot: MvccSnapshot[applied={T|T < 7316595346627928064}]
I20260809 12:22:40.875847 21811 catalog_manager.cc:1347] Generated new cluster ID: 4c13ed19ac464422b2cd533e43ff2265
I20260809 12:22:40.875953 21811 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 12:22:40.876142 21822 mvcc.cc:205] Tried to move back new op lower bound from 7316595346627928064 to 7316595346446372864. Current Snapshot: MvccSnapshot[applied={T|T < 7316595346627928064}]
I20260809 12:22:40.877578 21797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "82171170d65740648e6eb14e2fe2c9c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82171170d65740648e6eb14e2fe2c9c4" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40117 } } peers { permanent_uuid: "018b8ed1068643eeb0cd290105fc749a" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } } peers { permanent_uuid: "68c457b0bd4e440194ea6ef9bcac91ab" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } } }
I20260809 12:22:40.877848 21797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4 [sys.catalog]: This master's current role is: LEADER
I20260809 12:22:40.888830 21658 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 018b8ed1068643eeb0cd290105fc749a: Neither blocks nor log segments found. Creating new log.
I20260809 12:22:40.894081 21658 log.cc:824] T 00000000000000000000000000000000 P 018b8ed1068643eeb0cd290105fc749a: Log is configured to *not* fsync() on all Append() calls
I20260809 12:22:40.895145 21776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 68c457b0bd4e440194ea6ef9bcac91ab [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "82171170d65740648e6eb14e2fe2c9c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82171170d65740648e6eb14e2fe2c9c4" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40117 } } peers { permanent_uuid: "018b8ed1068643eeb0cd290105fc749a" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } } peers { permanent_uuid: "68c457b0bd4e440194ea6ef9bcac91ab" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } } }
I20260809 12:22:40.895432 21776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 68c457b0bd4e440194ea6ef9bcac91ab [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:22:40.934864 21811 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 12:22:40.936728 21811 catalog_manager.cc:1504] Loading token signing keys...
I20260809 12:22:40.950492 21811 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4: Generated new TSK 0
I20260809 12:22:40.951519 21811 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 12:22:40.918780 21456 master_runner.cc:186] Error getting master registration for 127.19.183.253:39393: OK, instance_id { permanent_uuid: "018b8ed1068643eeb0cd290105fc749a" instance_seqno: 1786278160071156 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 12:22:40.956447 21456 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 12:22:40.959156 21658 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 018b8ed1068643eeb0cd290105fc749a: No bootstrap required, opened a new log
I20260809 12:22:40.976956 21658 raft_consensus.cc:348] T 00000000000000000000000000000000 P 018b8ed1068643eeb0cd290105fc749a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82171170d65740648e6eb14e2fe2c9c4" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40117 } } peers { permanent_uuid: "018b8ed1068643eeb0cd290105fc749a" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } } peers { permanent_uuid: "68c457b0bd4e440194ea6ef9bcac91ab" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } }
I20260809 12:22:40.977427 21658 raft_consensus.cc:729] T 00000000000000000000000000000000 P 018b8ed1068643eeb0cd290105fc749a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 018b8ed1068643eeb0cd290105fc749a, State: Initialized, Role: FOLLOWER
I20260809 12:22:40.978021 21658 consensus_queue.cc:260] T 00000000000000000000000000000000 P 018b8ed1068643eeb0cd290105fc749a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82171170d65740648e6eb14e2fe2c9c4" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40117 } } peers { permanent_uuid: "018b8ed1068643eeb0cd290105fc749a" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } } peers { permanent_uuid: "68c457b0bd4e440194ea6ef9bcac91ab" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } }
I20260809 12:22:40.984967 21839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 018b8ed1068643eeb0cd290105fc749a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82171170d65740648e6eb14e2fe2c9c4" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40117 } } peers { permanent_uuid: "018b8ed1068643eeb0cd290105fc749a" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } } peers { permanent_uuid: "68c457b0bd4e440194ea6ef9bcac91ab" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } } }
I20260809 12:22:40.985350 21839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 018b8ed1068643eeb0cd290105fc749a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:22:40.987505 21622 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 018b8ed1068643eeb0cd290105fc749a [term 0 FOLLOWER]: Advancing to term 1
I20260809 12:22:40.991827 21622 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 018b8ed1068643eeb0cd290105fc749a [term 1 FOLLOWER]: Refusing update from remote peer 82171170d65740648e6eb14e2fe2c9c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 12:22:40.987684 21623 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "82171170d65740648e6eb14e2fe2c9c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "018b8ed1068643eeb0cd290105fc749a"
I20260809 12:22:40.992837 21797 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "018b8ed1068643eeb0cd290105fc749a" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 12:22:40.995781 21623 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 018b8ed1068643eeb0cd290105fc749a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 82171170d65740648e6eb14e2fe2c9c4 in term 1.
I20260809 12:22:40.988181 21658 sys_catalog.cc:564] T 00000000000000000000000000000000 P 018b8ed1068643eeb0cd290105fc749a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 12:22:41.025576 21845 mvcc.cc:205] Tried to move back new op lower bound from 7316595346627928064 to 7316595346446372864. Current Snapshot: MvccSnapshot[applied={T|T < 7316595346627928064}]
I20260809 12:22:41.035457 21842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 018b8ed1068643eeb0cd290105fc749a [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "82171170d65740648e6eb14e2fe2c9c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82171170d65740648e6eb14e2fe2c9c4" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40117 } } peers { permanent_uuid: "018b8ed1068643eeb0cd290105fc749a" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } } peers { permanent_uuid: "68c457b0bd4e440194ea6ef9bcac91ab" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } } }
I20260809 12:22:41.035889 21842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 018b8ed1068643eeb0cd290105fc749a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:22:41.036221 21839 sys_catalog.cc:455] T 00000000000000000000000000000000 P 018b8ed1068643eeb0cd290105fc749a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 82171170d65740648e6eb14e2fe2c9c4. Latest consensus state: current_term: 1 leader_uuid: "82171170d65740648e6eb14e2fe2c9c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82171170d65740648e6eb14e2fe2c9c4" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40117 } } peers { permanent_uuid: "018b8ed1068643eeb0cd290105fc749a" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } } peers { permanent_uuid: "68c457b0bd4e440194ea6ef9bcac91ab" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } } }
I20260809 12:22:41.036458 21839 sys_catalog.cc:458] T 00000000000000000000000000000000 P 018b8ed1068643eeb0cd290105fc749a [sys.catalog]: This master's current role is: FOLLOWER
I20260809 12:22:41.063097 21863 catalog_manager.cc:795] Waiting for catalog manager background task thread to start: Service unavailable: Catalog manager is not initialized. State: Starting
W20260809 12:22:41.089221 20191 mini_kdc.cc:348] Time spent kdestroy: real 0.127s user 0.000s sys 0.035s
W20260809 12:22:41.130184 21663 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:41.118909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37516 (local address 127.19.183.253:39393)
0809 12:22:41.119111 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:41.119117 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:41.127502 (+ 8385us) server_negotiation.cc:407] Connection header received
0809 12:22:41.127940 (+ 438us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:41.127950 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:41.128091 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:41.128179 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:41.129891 (+ 1712us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37516: BlockingRecv error: recv got EOF from 127.0.0.1:37516 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 12:22:41.130901 21771 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:41.118889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57382 (local address 127.19.183.254:40117)
0809 12:22:41.119103 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:41.119110 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:41.128856 (+ 9746us) server_negotiation.cc:407] Connection header received
0809 12:22:41.129354 (+ 498us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:41.129363 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:41.129448 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:41.129541 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:41.130641 (+ 1100us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57382: BlockingRecv error: recv got EOF from 127.0.0.1:57382 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260809 12:22:41.132927 21768 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:41.119752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40690 (local address 127.19.183.252:36651)
0809 12:22:41.119982 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:41.119991 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:41.131088 (+ 11097us) server_negotiation.cc:407] Connection header received
0809 12:22:41.131465 (+ 377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:41.131473 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:41.131577 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:41.131665 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:41.132622 (+ 957us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40690: BlockingRecv error: recv got EOF from 127.0.0.1:40690 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260809 12:22:41.140671 21663 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:41.136949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37522 (local address 127.19.183.253:39393)
0809 12:22:41.137199 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:41.137208 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:41.139230 (+ 2022us) server_negotiation.cc:407] Connection header received
0809 12:22:41.139469 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:41.139478 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:41.139594 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:41.139704 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:41.140412 (+ 708us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37522: BlockingRecv error: recv got EOF from 127.0.0.1:37522 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
W20260809 12:22:41.152478 21663 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:41.148028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37526 (local address 127.19.183.253:39393)
0809 12:22:41.148235 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:41.148243 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:41.151214 (+ 2971us) server_negotiation.cc:407] Connection header received
0809 12:22:41.151403 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:41.151408 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:41.151482 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:41.151589 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:41.152290 (+ 701us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37526: BlockingRecv error: recv got EOF from 127.0.0.1:37526 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260809 12:22:41.163333 21663 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:41.157180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37540 (local address 127.19.183.253:39393)
0809 12:22:41.157378 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:41.157386 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:41.162019 (+ 4633us) server_negotiation.cc:407] Connection header received
0809 12:22:41.162215 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:41.162219 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:41.162292 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:41.162399 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:41.163115 (+ 716us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37540: BlockingRecv error: recv got EOF from 127.0.0.1:37540 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 12:22:41.172015 21663 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:41.169925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37552 (local address 127.19.183.253:39393)
0809 12:22:41.170204 (+ 279us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:41.170212 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:41.170734 (+ 522us) server_negotiation.cc:407] Connection header received
0809 12:22:41.170940 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:41.170949 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:41.171011 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:41.171112 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:41.171754 (+ 642us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37552: BlockingRecv error: recv got EOF from 127.0.0.1:37552 (error 108)
Metrics: {"server-negotiator.queue_time_us":125}
W20260809 12:22:41.185341 21663 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:41.183282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37554 (local address 127.19.183.253:39393)
0809 12:22:41.183494 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:41.183500 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:41.184038 (+ 538us) server_negotiation.cc:407] Connection header received
0809 12:22:41.184256 (+ 218us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:41.184260 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:41.184324 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:41.184431 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:41.185127 (+ 696us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37554: BlockingRecv error: recv got EOF from 127.0.0.1:37554 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
W20260809 12:22:41.206933 21663 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:41.204633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37564 (local address 127.19.183.253:39393)
0809 12:22:41.204862 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:41.204871 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:41.205431 (+ 560us) server_negotiation.cc:407] Connection header received
0809 12:22:41.205680 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:41.205685 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:41.205766 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:41.205888 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:41.206664 (+ 776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37564: BlockingRecv error: recv got EOF from 127.0.0.1:37564 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
W20260809 12:22:41.244979 21663 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:41.242619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37580 (local address 127.19.183.253:39393)
0809 12:22:41.242870 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:41.242880 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:41.243470 (+ 590us) server_negotiation.cc:407] Connection header received
0809 12:22:41.243757 (+ 287us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:41.243761 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:41.243836 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:41.243944 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:41.244711 (+ 767us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37580: BlockingRecv error: recv got EOF from 127.0.0.1:37580 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260809 12:22:41.315804 21663 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:41.313057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55568 (local address 127.19.183.253:39393)
0809 12:22:41.313361 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:41.313370 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:41.314049 (+ 679us) server_negotiation.cc:407] Connection header received
0809 12:22:41.314384 (+ 335us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:41.314389 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:41.314506 (+ 117us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:41.314630 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:41.315580 (+ 950us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55568: BlockingRecv error: recv got EOF from 127.0.0.1:55568 (error 108)
Metrics: {"server-negotiator.queue_time_us":105}
W20260809 12:22:41.450047 21663 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:41.448044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55576 (local address 127.19.183.253:39393)
0809 12:22:41.448233 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:41.448237 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:41.448663 (+ 426us) server_negotiation.cc:407] Connection header received
0809 12:22:41.448837 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:41.448841 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:41.448916 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:41.449012 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:41.449796 (+ 784us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55576: BlockingRecv error: recv got EOF from 127.0.0.1:55576 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260809 12:22:41.462819 21797 sys_catalog.cc:455] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "82171170d65740648e6eb14e2fe2c9c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "82171170d65740648e6eb14e2fe2c9c4" member_type: VOTER last_known_addr { host: "127.19.183.254" port: 40117 } } peers { permanent_uuid: "018b8ed1068643eeb0cd290105fc749a" member_type: VOTER last_known_addr { host: "127.19.183.253" port: 39393 } } peers { permanent_uuid: "68c457b0bd4e440194ea6ef9bcac91ab" member_type: VOTER last_known_addr { host: "127.19.183.252" port: 36651 } } }
I20260809 12:22:41.463138 21797 sys_catalog.cc:458] T 00000000000000000000000000000000 P 82171170d65740648e6eb14e2fe2c9c4 [sys.catalog]: This master's current role is: LEADER
W20260809 12:22:41.711738 21663 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 12:22:41.709912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55582 (local address 127.19.183.253:39393)
0809 12:22:41.710124 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:41.710130 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:41.710454 (+ 324us) server_negotiation.cc:407] Connection header received
0809 12:22:41.710633 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:41.710637 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:41.710729 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:41.710826 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 12:22:41.711556 (+ 730us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55582: BlockingRecv error: recv got EOF from 127.0.0.1:55582 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 12:22:41.725848 21791 catalog_manager.cc:1259] Loaded cluster ID: 4c13ed19ac464422b2cd533e43ff2265
I20260809 12:22:41.725950 21791 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 68c457b0bd4e440194ea6ef9bcac91ab: loading cluster ID for follower catalog manager: success
I20260809 12:22:41.728639 21791 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 68c457b0bd4e440194ea6ef9bcac91ab: acquiring CA information for follower catalog manager: success
I20260809 12:22:41.730557 21791 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 68c457b0bd4e440194ea6ef9bcac91ab: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 12:22:42.067531 21863 catalog_manager.cc:1259] Loaded cluster ID: 4c13ed19ac464422b2cd533e43ff2265
I20260809 12:22:42.067682 21863 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 018b8ed1068643eeb0cd290105fc749a: loading cluster ID for follower catalog manager: success
I20260809 12:22:42.070240 21863 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 018b8ed1068643eeb0cd290105fc749a: acquiring CA information for follower catalog manager: success
I20260809 12:22:42.072081 21863 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 018b8ed1068643eeb0cd290105fc749a: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 12:22:42.235213 21956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:42.228494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55598 (local address 127.19.183.253:39393)
0809 12:22:42.229711 (+ 1217us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:42.229717 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:42.229738 (+ 21us) server_negotiation.cc:407] Connection header received
0809 12:22:42.229817 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:42.229820 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:42.229968 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:42.230109 (+ 141us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:22:42.230605 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:42.231489 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:42.232836 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:42.233319 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:42.233465 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:42.233983 (+ 518us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:22:42.234540 (+ 557us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:22:42.234741 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:42.234806 (+ 65us) server_negotiation.cc:299] Negotiation successful
0809 12:22:42.234897 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1069,"thread_start_us":871,"threads_started":1}
I20260809 12:22:42.247658 21964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:42.241475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54322 (local address 127.19.183.252:36651)
0809 12:22:42.242012 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:42.242018 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:42.242374 (+ 356us) server_negotiation.cc:407] Connection header received
0809 12:22:42.242559 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:42.242563 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:42.242789 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:42.242975 (+ 186us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:22:42.243398 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:42.244142 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:42.245425 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:42.245815 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:42.245952 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:42.246446 (+ 494us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:22:42.247035 (+ 589us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:22:42.247207 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:42.247266 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 12:22:42.247349 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":285,"threads_started":1}
I20260809 12:22:42.274823 21956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:42.254512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55608 (local address 127.19.183.253:39393)
0809 12:22:42.254733 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:42.254742 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:42.266799 (+ 12057us) server_negotiation.cc:407] Connection header received
0809 12:22:42.267109 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:42.267113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:42.267201 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:42.267274 (+ 73us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:22:42.268362 (+ 1088us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:42.269497 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:42.272416 (+ 2919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:42.273083 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:42.273184 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:42.273949 (+ 765us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:22:42.274079 (+ 130us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:22:42.274151 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:42.274407 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 12:22:42.274507 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 12:22:42.275105 21971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:42.254071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58226 (local address 127.19.183.254:40117)
0809 12:22:42.265786 (+ 11715us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:42.265795 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:42.266797 (+ 1002us) server_negotiation.cc:407] Connection header received
0809 12:22:42.266935 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:42.266941 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:42.267199 (+ 258us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:42.267384 (+ 185us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:22:42.268254 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:42.269422 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:42.272156 (+ 2734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:42.272822 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:42.272947 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:42.273642 (+ 695us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:22:42.274079 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:22:42.274182 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:42.274672 (+ 490us) server_negotiation.cc:299] Negotiation successful
0809 12:22:42.274791 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":11546,"thread_start_us":11725,"threads_started":1}
I20260809 12:22:42.278116 21964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 12:22:42.260296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54338 (local address 127.19.183.252:36651)
0809 12:22:42.260815 (+ 519us) server_negotiation.cc:206] Beginning negotiation
0809 12:22:42.260824 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 12:22:42.271023 (+ 10199us) server_negotiation.cc:407] Connection header received
0809 12:22:42.271355 (+ 332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 12:22:42.271361 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 12:22:42.271448 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 12:22:42.271511 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 12:22:42.272366 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:42.273413 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:42.275582 (+ 2169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 12:22:42.276129 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 12:22:42.276205 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 12:22:42.276902 (+ 697us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 12:22:42.277027 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 12:22:42.277105 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 12:22:42.277721 (+ 616us) server_negotiation.cc:299] Negotiation successful
0809 12:22:42.277808 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340}
I20260809 12:22:42.280938 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 21456
I20260809 12:22:42.282172 21543 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:22:42.423254 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 21456
I20260809 12:22:42.453562 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 21556
I20260809 12:22:42.454785 21652 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:22:42.561801 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 21556
I20260809 12:22:42.589677 20191 external_mini_cluster.cc:1636] Attempting to check leaks for /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu pid 21669
I20260809 12:22:42.590890 21759 generic_service.cc:196] Checking for leaks (request via RPC)
I20260809 12:22:42.700317 20191 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskqUnAOm/build/asan/bin/kudu with pid 21669
2026-08-09T12:22:42Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (5257 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (10376 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (387837 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST